Commit Graph

1641 Commits

Author SHA1 Message Date
Ziye Yang
48701bd955 lib/event: support ABI compatibility for spdk_app_opts.
This patch is used to support ABI compatability issue for spdk_app_opts.

Fixes #1484

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I6fed777fa15b367f7c3706b8f218d86d56493906
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5330
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
2020-12-16 08:40:14 +00:00
Jim Harris
17afd268df nvme: call correct remove_cb when device is removed
When a device is removed, we should use the remove_cb
that was specified when the device was originally probed
and attached, if one was set.

Also add a new spdk_nvme_ctrlr_set_remove_cb API.  This
can be used for cases where a different remove_ctx is
desired than was specified for the probe call.  This
also enables setting a remove_cb when using connect APIs
which do not have a way currently to provide a remove_cb.

This also requires fixing the bdev nvme module, which
was depending on the previously errant behavior.

Fixes issue #1715.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id848b39040099ff7a21fe57ea6b194a8c25ae015

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5510
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-12-14 14:34:42 +00:00
Ziye Yang
3b16c6ddc2 lib/nvmf: support ABI compatibility for spdk_nvmf_transport_opts
This patch is used to support ABI compatibility related with
spdk_nvmf_transport_opts structure. We add a field opts_size in
spdk_nvmf_transport_opts and change the related two functions.

Fixes issue: 1485

Change-Id: Ifed3dc482bbc8fb54eb7089f7a1931718682f214
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5293
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
2020-12-11 11:10:16 +00:00
Jin Yu
da625683a6 bdev: virtio blk read max size and segs
The backend device such as virtio-blk or virtio-scsi
may support the SIZE_MAX and SEG_MAX. Then SPDK needs
to split the big IO. Add this feature in bdev.

Change-Id: I2442e14121ccf141682964425e96382fec482af3
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4600
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-10 14:39:45 +00:00
Jim Harris
3795c1cfd8 event: replace master_core with main_core in spdk_app_opts
master_core will still be available but deprecated.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6fe9eb7fecb2919795c439a02cb7d9d06a2774d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5358
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-03 09:41:07 +00:00
Jim Harris
fe137c8970 env: replace master_core with main_core
This helps us remove master/slave terminology from
SPDK and is aligned with similar changes made recently
in DPDK.

While updating nvme/identify to use the new member
name, also replace g_master_core there with g_main_core.
Other nvme utility usage of "master_core" will be updated
in a later patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0ec4e3e9b644bec21b3729809bf5c4d35b10837f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5351
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-03 09:41:07 +00:00
Jim Harris
131e617191 nvmf: remove usage of "whitelist"
NVMf subsystems can have certain hosts that are allowed.
We were previously using the term "whitelist" to refer
to those hosts.  Replace that usage as part of removing
this term from the rest of SPDK.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2878352d83f3911aa8b59629061b818c7f1e5c96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5282
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-12-03 09:41:07 +00:00
Jim Harris
9c2b3b3535 event: add pci_allowed/pci_blocked to spdk_app_opts
The old terms pci_whitelist/pci_blacklist are now
deprecated.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6350a6dbe21ceb5965b30241899eead651834dca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5280
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-03 09:41:07 +00:00
Jim Harris
4a6a282411 env: add pci_allowed/pci_blocked to spdk_env_opts
The old pci_whitelist/pci_blacklist are now deprecated.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9fddec0c90691dd385eb21d13be849247f144889
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5279
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-03 09:41:07 +00:00
Jacek Kalwas
296a6d9838 bdev: extend interface with get module ctx
Currently only nvme bdev module implements this interface. Bdev module
context (in this case spdk_nvme_ctrlr opaque handle) allows for nvme
interface usage for additional management.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I6302c9229d5f7f294a3c1472d9e8dc1519637ffb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-12-01 10:20:56 +00:00
Ben Walker
af07b82fa2 nvmf: The maximum number of namespaces a subsystem may contain must now
always be specified

Previously the parameter was optional and the size could be increased as
necesary. Now, it is required and a hard maximum. Later another function
could be added to dynamically increase or reduce this number if
necessary.

Change-Id: I3524ac737a6b592b4f6ce14ea48d3742a352c70f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-12-01 10:20:43 +00:00
Jacek Kalwas
86e8085d58 env: describe enumerate function detail behavior
2s asynchronous delay is introduced to avoid race conditions between
user space software initialization and in-kernel device handling for
newly inserted devices. Subsequent enumerate call after the delay
shall allow for a successful device attachment.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I273111ae7a588ce5e15e593bbdfb223777c19071
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5060
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-11-26 10:15:37 +00:00
Jacek Kalwas
4793391448 env: move attach cb description where typedef is specified
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I148b4b1608b4287add0e7d3b88b676950d44a9a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5059
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-11-25 08:52:43 +00:00
Naresh Gottumukkala
0d98a94901 nvmf: Allow asynchronous nvmf transport destroy.
As part of FC transport destroy, FC LLD (Low level Driver) needs to
to do its cleanup which cannot be completed synchronously. So
allow transport destroy to be asynchronous.

FC transport code to use this functionality will be pushed shortly.

Signed-off-by: Naresh Gottumukkala <raju.gottumukkala@broadcom.com>
Change-Id: I104cf7d131e18199abdcf0651df261fe41d666ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5180
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-11-24 08:21:27 +00:00
Niklas Cassel
23a1916935 nvme: add function to get the max zone append data transfer size
The zone append command, which is part of the Zoned Namespace Command Set,
has a maximum data transfer size that can be less than or equal to mdts.

Since zone append commands will not be allowed to be split, the user has
to be able to get the maximum zone append data transfer size. Add a
function that returns this limit.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I9da2672ea8a307ff62251c069a42f7540765e08b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5140
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-11-20 11:00:18 +00:00
Niklas Cassel
38d59d8b5a nvme: add function to check if controller supports zone append
Zone append is an optional command in the Zoned Namespace Command Set.
Add a convenience function to check if the controller supports the zone
append command.

The ratified NVMe TP 4056 added a CSI field (in cdw14) to the Get Log Page
command. However, since there already exist two public functions to get a
log page (spdk_nvme_ctrlr_cmd_get_log_page() and
spdk_nvme_ctrlr_cmd_get_log_page_ext()), avoid creating a third one for
now, since nvme_ctrlr_get_zns_cmd_and_effects_log() itself can leverage
one of the existing public functions.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I99516dbac8db6714488b4d6cabe64c27f46d6153
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5078
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-11-20 11:00:18 +00:00
Niklas Cassel
35596d34b0 nvme: make enum spdk_nvme_ctrlr_flags more readable
Make enum spdk_nvme_ctrlr_flags more readable.
Other enums in spdk, e.g. enum accel_capability,
and enum NVME_RDMA_COMPLETION_FLAGS, already define
the enum values using bit shifts.

Do the same for enum spdk_nvme_ctrlr_flags.

No functional change intended.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ida7d7e4ff7f4cffce85cb2d6ea2bc0168875fdb8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5158
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-11-20 11:00:18 +00:00
Niklas Cassel
bd9db7f8c2 nvme: add missing param in doc for spdk_nvme_ctrlr_get_max_xfer_size()
Add missing param keyword in documentation for
spdk_nvme_ctrlr_get_max_xfer_size().

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I0e4242b41b50a12da5a79e6fef135e5f3046ac6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5139
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-11-20 11:00:18 +00:00
Nick Connolly
b869772938 Include: Test for __linux__
Explicitly test for __linux__ which is clearly the intent and
matches the behaviour described in the comments.

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: I498476f4a214891b897609924e018c5b132afaf2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5143
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-11-19 09:53:21 +00:00
Naresh Gottumukkala
ccd96ead65 nvmf: Allow nvmf_transport_qpair_fini to complete asynchronously.
As part of nvmf_transport_qpair_fini, FC transport needs to cleanup
all the resources used by that QPair on the shared hardware. This
hardware cleanup is asynchronous in nature.

FC transport code to use this functionality will be pushed shortly.

Change-Id: I5606a33dff45971badd74e0cc087b132b56af076
Signed-off-by: Naresh Gottumukkala <raju.gottumukkala@broadcom.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5100
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
2020-11-18 11:28:02 +00:00
yidong0635
d73077b84a lib/jsonrpc: Add a new API to send response for writing bool result.
There are many duplicated codes about sending response for writing bool result.
That we need a function to do this.
Then we can reduce many codes.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic439111b1e9ca1013f8c657ab925f0c27a7be699
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5033
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-11-16 15:08:47 +00:00
Madhu Adav MJ
fce9428772 nvmf: Async event support for discovery log change
Added asynchronous event notices for discovery log change
as per nvme fabrics spec 1.1. This allows a host with persistent
connection to discovery controller to automatically connect to any
new subsystem available to the host automatically.

According to nvme fabrics spec 1.1, if the connect command specifies a
non-zero keep alive timer value and the discovery controller does not
support asynchronous events then we need to return Connect Invalid.
Since SPDK does not implement this check instead added support
for asynchronous events in discovery controller.

Change-Id: I4cade5f7d24826ce97a2fa2b4ca688a1d728c1db
Signed-off-by: Madhu Adav MJ <madhu.adav@flipkart.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
2020-11-11 00:53:57 +00:00
Jacek Kalwas
7229b8e1c6 env: update description for enumerate function
During enumeration all registered pci devices with exposed access to
userspace are getting probed internally unless not explicitly specified
on denylist. Because of that it becomes not possible to either use such
devices with another application or unbind the driver (e.g. vfio).

Change-Id: I9e84ed1d5dc82db75adcb18936cb6d702ee74d78
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4962
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-11-02 11:19:43 +00:00
Jim Harris
ddf86600bb nvme: continue probing ctrlrs even if one fails
It is possible that a single probe_ctx could be used
to probe multiple newly attached nvme controllers.  If
one of those controllers is removed during this process,
the rest of the controllers do not get probed and can
even get stuck in a zombie state.

It is better to just continue with probing the rest of
the controllers.

Fixes issue #1611.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4156ee8b50e8d52cfeee7224f210a58bb773e939

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4945
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 08:11:17 +00:00
Jim Harris
c31ad66893 event: deprecate opts.config_file member
Just always put the config file name in json_config_file,
since we now only support JSON.

If user specifies both -c and --json, it will just take
the latter of the two.  This is similar to if the user
specified --json twice.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idc21d73acf0e190eda57a7b0c5d9bcfa14e87030
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4858
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-10-26 22:25:47 +00:00
Tomasz Zawadzki
214b9847d0 version: SPDK 21.01 pre
This is first commit that should go into latest SPDK
after the code freeze for SPDK 20.10.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie7dc39553206ba747dc83ab6af7ddc2fc25d71eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4854
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-23 23:21:14 +00:00
Liu Xiaodong
4bf6e4bb32 interrupt: apply fd_group in thd and reactor
Each reactor and each thread is assigned with one fd group.
At the same time, each thread is treated as one interrupt source
registered into its corresponding reactor.
The egrp function for reacotr is the only block point waiting
for events.

Change-Id: Id092e66591b07b445342f8ae16dc218d28887427
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
c5d1e7f171 interrupt: new fd_group in lib/util
Interrupt is aiming to support interrupt
mode in SPDK application framework.
fd_group is implemented by epoll on linux.
fd_group can be inserted into thread and reactor
to do epoll_wait internally in order to avoid polling

Change-Id: I9077ad648a97fbd68fd46f43de2e16440bedab0b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4268
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Simon A. F. Lund
54219afe5a nvme/spec: add accessors to, and commented members of, zone-descriptors
The struct-accessors are added and named matching the fields defined in
the spec. to be used by the fio_plugin/nvme and other consumers of the
driver-layer. Comments to be consumed doc-generators as well as human
readers of the header-file.

The identify example is updated with the change.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I8d6cb82e095c5dcbc06fe892e17ce83dc0062735
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4835
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:01 +00:00
Simon A. F. Lund
6fafaa5dc6 nvme/spec: add ZNS zone states and zone types
Added to be used by the fio_plugin/nvme and other consumers of the
driver-layer.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I945b0aa93c675ad381ce0a83883c41688b549aee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4834
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:01 +00:00
paul luse
17bb748a60 accel: Move non-engine specific batch to the accel_fw layer
The new design:

* Supports a generic batching capability in the accel_fw layer
that keeps track of hw accelerated vs sw commands based on
the capabilities of the engine and processes sw commands in the
generic layer while sending a list of commands (not a batch)
to the engines for processing.

* Batch completions are managed via the generic layer, when using
the accel_fw the engines only process commands. With DSA however,
if a list of commands is sent down it will use the DSA public
API to create and send batches but will rely on the generic layer
to complete the batch task itself. When using DSA directly, batching
works as usual (DSA handles batch completion).

* The engine function tables were greatly simplified by replacing
all of the individual entries (copy, fill, crc32c, etc) with one
`submit_tasks` function that is used to both send lists of tasks
for batches or just one task for single shot API.

* Internally batching is now used to re-submit tasks that were queued
for flow control reasons.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I99c28751df32017c43490a90f4904bdabe79a270
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2020-10-22 22:43:28 +00:00
Changpeng Liu
c0bf93145a env/memory: return the memory virtual address's file descriptor and offset
For virtio-user library and coming vfio-user feature, the client needs
to send the memory file descriptors to target so that the two processes
can setup shared memory region to do data processing without memory copy.
Currently virtio-user will read /proc/self/maps to get memory file descriptor,
since DPDK already provides this such APIs, so here we can just use it,
for existing virtio-user library we may replace it with the new added
API.

Change-Id: Icfeae465d53826d0c8d1b335287634b03cd174aa
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-21 20:47:01 +00:00
Tomasz Zawadzki
344bb69312 event: remove printing legacy config for apps
Remove spdk_app_get_running_config() that allowed printing
legacy configuration by the apps.
Along with usr1_handler callback that was used to call that
function.

It was only used in iscsi_tgt so it is removed there.

The app_repeat test was using SIGUSR1 to trigger a
spdk_app_stop/spdk_app_start cycle.  But we can use
SIGTERM for that instead.  While here, do a bit of
cleanup in the app_repeat test app.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie8449e8bcd979b237c68ef94f50fb7a3cb6c19db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4750
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-21 20:44:47 +00:00
Niklas Cassel
6dec7623be nvme: add support for ZNS zone management receive command
Add support for the ZNS zone management receive command.
An internal nvme_zns_zone_mgmt_recv() function is created
that matches the parameters of the zone management receive
function in the ZNS specification.

Convenience functions are provided for the following
Zone Receive Action: Report Zones.

Zone Receive Actions not implemented: Extended Report
Zones.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I23589a602336da5dffccec7230d07026a868e81b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4793
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-21 12:18:44 +00:00
Niklas Cassel
be3ff9c0b5 nvme: add function to get ZNS num zones
Add a function to get the number of zones for a zoned namespace.

Since the ZNS specification does not allow zone size == 0,
divide by zero should not be possible on a spec compliant drive.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I15e0ac8a72e244e248b7fb44f7156b182ecd98b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4792
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-21 12:18:44 +00:00
Niklas Cassel
6abee9a205 nvme: add function to get ZNS zone size
Add a function to get the zone size of a zoned namespace.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I89429fa2fd36307a8b34bdf91e3251c68596a48c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
2020-10-21 12:18:44 +00:00
Tomasz Zawadzki
28ebb8cc78 bdev_module: remove config_text
This patch removes function for bdev modules to
present options of the bdevs.

blob_bdev.h refers to the spdk_bdev_module, so would need
to be bumped too.
At this time spdk_bdev_module is left unchanged to prevent
that.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3cacb087c998d928c5d8c2722b7f041d82bb43f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4748
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-21 02:25:45 +00:00
Shuhei Matsumoto
d8f4bbeb43 lib/nvme: Add spdk_nvme_detach_async() and spdk_nvme_detach_poll_async()
Add two new public APIs, spdk_nvme_detach_async() and
spdk_nvme_detach_poll_async() to detach multiple controllers in parallel
as a simple manner to users.

Hold the target controller to nvme_ctrlr_detach_ctx because users
will free any object which held it after returning spdk_nvme_detach_async().

spdk_nvme_detach_ctx holds all nvme_ctrlr_detach_ctx in a sequence
by linked list.

spdk_nvme_detach_ctx has a boolean variable polling_started to prevent
the user from calling spdk_nvme_detach_async() while
spdk_nvme_detach_poll_async() is called repeatedly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>i
Change-Id: Ib049c19f7ef24410b963fd5c777a21184f3012d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4758
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 16:55:55 +00:00
Niklas Cassel
67475a58f4 nvme: add support for ZNS zone management send command
Add support for the ZNS zone management send command.
An internal nvme_zns_zone_mgmt_send() function is created
that matches the parameters of the zone management send
function in the ZNS specification.

Convenience functions are provided for the following
Zone Send Actions: Close Zone, Finish Zone, Open Zone,
Reset Zone, and Offline Zone.

Zone Send Actions not implemented: Set Zone Descriptor
Extension.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I9cc04776c249d2b9c515438b81a0f06aea4198fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-20 12:18:56 +00:00
Niklas Cassel
3be2935701 nvme: move ZNS specific functions to their own header file
Move ZNS specific functions to their own header file:
include/spdk/nvme_zns.h.
This is similar to how it's done for e.g. NVMe-oF with
include/spdk/nvmf.h.

This avoids cluttering the nvme.h header with ZNS specific functions.

A SPDK program that wants to use the spdk_nvme_zns_* functions will
need to include include/spdk/nvme_zns.h, however, this is similar
to how e.g. spdk_nvmf namespaced functions work.
This change is safe to do now, since there is no SPDK release with
ZNS so far.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: If39fab4e7a6924cc3bd67aa0c03e9c6de5f8a324
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4681
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-20 12:18:56 +00:00
Shuhei Matsumoto
3ea2bffb9e blob/bdev: Add function pointer get_base_bdev to struct spdk_bs_dev
spdk_bdev_create_bs_dev_ext() gets not bdev pointer but bdev name
as an argument, and hence vbdev_lvs_create() will get bdev name
accordingly.

However after completing spdk_bdev_create_bs_dev_ext(),
vbdev_lvs_create() has to get bdev pointer from the created bs_dev.

Hence add a function pointer get_base_bdev to struct spdk_bs_dev
and set it to bdev_blob_get_base_bdev() at initialization.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idef0663ace85db0269442212014286669c150069
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4706
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Shuhei Matsumoto
6a72c19ee2 blob/bdev: Add an API spdk_bdev_create_bs_dev_ext()
spdk_bdev_open_ext() requires the caller to use bdev_event_cb_t
and bdev_event_cb_t is more extensible than bdev_remove_cb_t.
Hence use bdev_event_t as an argument.

spdk_bdev_open_ext() calls spdk_bdev_get_by_name() inside and
spdk_bdev_create_bs_dev_ext() calls spdk_bdev_open_ext() inside.
The caller needs to know if the spdk_bdev_get_by_name() succeeded.
Hence spdk_bdev_create_bs_dev_ext() returns return code by return
value and returns the created bs_dev by double pointer.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1c225bfb66db036439c69c459f39c86684d8a540
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Tomasz Zawadzki
a8de86e9aa lib/accel: remove legacy config support for accel module register
Since legacy config is being removed, SPDK_ACCEL_MODULE_REGISTER
macro no longer needs field corresponding to processing this
type of configuration.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I76ca02487a5fb8f9ecc9d33ac6fa514e7226a03b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 02:42:16 +00:00
Darek Stojaczyk
c77c655987 json: add spdk_json_free_object()
After decoding a JSON object we had to free the parsed
strings one-by-one. Not anymore.

Change-Id: I819f1d533e397aa9babca58b5500c38ac01a963d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2753
Reviewed-by: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-19 10:02:10 +00:00
Darek Stojaczyk
f766d1e478 nvmf: let transports parse transport opts from JSON themselves
There are more transport on the way and we don't want to add
all their various opts into the single, generic structure.
We'll pass the JSON structure to transports instead. Then
the transport code can custom pull from the JSON any param
it wants.

To complement that, transports will now also have their own
JSON config dump callback. This was only done in the generic
nvmf.c so far, with conditions for RDMA and TCP.

Change-Id: I33115a8d56cec829b1c51311a318e0333cc64920
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-19 09:55:05 +00:00
Tomasz Zawadzki
1a884e94e4 lib/log: do not use spdk_log_get_flag() in macro for log flags
Especially SPDK_INFOLOG() should not call spdk_log_get_flag() function,
instead rely on the log flag structure.

This caused performance degradation in places that used SPDK_INFOLOG
too extensively.

Due to nature of the macros, in the meantime new debug log in
iscsi was improperly named. This wasn't caught because of the
macros reliance on spdk_log_get_flag(), where if name wasn't
registered - no log was printed and no compilation occurred.
Going back to using the structures, gives immediate feedback
that there is no SPDK_LOG_ISCSI log flag.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I262eadb5aee29caa2a5cd719c82ca4963ea24576
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-16 08:14:59 +00:00
Shuhei Matsumoto
db8bd99588 bdev/part: Add spdk_bdev_part_base_construct_ext() to pass bdev_name
Add an new API spdk_bdev_part_base_construct_ext() to pass not bdev but
bdev_name to fix the race condition due to the time gap between
spdk_bdev_get_by_name() and spdk_bdev_open(). A pointer to a bdev is
valid only while the bdev is opened.

In the new API, spdk_bdev_get_by_name() is included in
spdk_bdev_part_base_construct_ext() and the caller has to know if
the bdev exists or not. Hence spdk_bdev_part_base_construct_ext()
returns return code and returns the created part object by the double
pointer.

Another critical change is that base is just freed if spdk_bdev_open_ext()
failed with -ENODEV. The reason is that if we call spdk_bdev_part_base_free()
for that case, the configuration is removed by the registered callback
and so bdev_examine() will not work.

The following patches will replace spdk_bdev_part_base_construct()
by spdk_bdev_part_base_construct_ext() for the corresponding bdev
modules.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2db027a159559c403cdfbd71800afba590b0f328
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-16 08:14:34 +00:00
Shuhei Matsumoto
bba9e8dbca bdev/part: Fix wrong doc about return value of spdk_bdev_part_base_construct()
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I133f08648b783d3faa8e1a8d3655812630106dab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4586
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-16 08:14:34 +00:00
Ben Walker
5efa3d61ff nvmf: Add spdk_nvmf_subsystem_disconnect_host
This will disconnect all connections to a subsystem from a given
host identified by HOSTNQN.

Change-Id: Ibc9cea1f08a58a05dbac3a0bb47df8d8a58e7c10
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4556
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-10-15 08:24:05 +00:00
Shuhei Matsumoto
a22d8a530f lib/nvmf: Add an API spdk_nvmf_subsystem_add_ns_ext() to pass bdev_name
Add an new API spdk_nvmf_subsystem_add_ns_ext() to pass not bdev but
bdev_name to fix the race condition due to the time gap between
spdk_bdev_get_by_name() and spdk_bdev_open(). A pointer to a bdev is
valid only while the bdev is opened.

spdk_bdev_open() has been replaced by spdk_bdev_open_ext() but the
issue still existed.

Update the corresponding unit tests accordingly.

Then replace the internal of spdk_nvmf_subsystem_add_ns() by
spdk_nvmf_subsystem_add_ns_ext() call.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifcaa2121129ef22d5e61c9a8f7c640ff37a64485
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4485
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-14 08:07:06 +00:00