In some cases we need to check if we are working with
SoftRoCE driver (rxe) since it doesn't support several features
such as SEND_WITH_INVAL and LAST_WQE_REACHED ibv async event.
vendor_id of this driver has been changed from 0 (it was not
initialized at all) to 0XFFFFFF in commit
0184afd15a141d7ce24c32c0d86a1e3ba6bc0eb3
So now we should check for both values.
Change-Id: I62ce4d68c11612a07fe97606658a5b79f1db2149
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5307
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There is a public API spdk_nvmf_qpair_get_listen_trid() and it can
get trid of the specified qpair safely for any transport including
pluggable transports.
The API was overlooked when implementing the multipath feature for
NVMe-oF target. trid pointer was added to struct spdk_nvmf_qpair and
was used to find a subsystem listener.
However, pluggable transports got seg. fault because trid of the
qpair was not set.
To avoid such segmentation fault for any transport, change
nvmf_ctrlr_create() to use spdk_nvmf_qpair_get_listen_trid().
The struct spdk_nvmf_qpair is located in the public header file,
and so leave the added trid for now. It will be deprecated eventually.
Reported-by: Jacek Kalwas <jacek.kalwas@intel.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5e0bd24bd58b6ffdf1352332a179a82682f1589f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5323
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
checks
Check if the subsystem is paused before checking whether the qpair is
active. The order of these checks doesn't really matter, but in the next
patch it will be more convenient to check the subsystem first.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ibc95e2578e9a95296f5fbc6023af3b542e954781
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This function is only called from one spot, so it's simpler to combine
it with that function.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I0a35e11edaf6c4446de530a3b0f538aa4a3238f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
always be specified
Previously the parameter was optional and the size could be increased as
necesary. Now, it is required and a hard maximum. Later another function
could be added to dynamically increase or reduce this number if
necessary.
Change-Id: I3524ac737a6b592b4f6ce14ea48d3742a352c70f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
As part of FC transport destroy, FC LLD (Low level Driver) needs to
to do its cleanup which cannot be completed synchronously. So
allow transport destroy to be asynchronous.
FC transport code to use this functionality will be pushed shortly.
Signed-off-by: Naresh Gottumukkala <raju.gottumukkala@broadcom.com>
Change-Id: I104cf7d131e18199abdcf0651df261fe41d666ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5180
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
POSIX defines PRId64/PRIu64/PRIx64 for printing 64-bit values in a
portable way. Replace a few references to %ld to remove the assumption
about the size of a long. Similarly, use %z with size_t arguments.
Where the value being printed is an unsigned 64-bit value, use PRIu64
instead of %ld.
Explicitly test for not __linux__ where that is the intent, rather
than testing for __FreeBSD__.
Cast pointer to uintptr_t before aligning it, rather than using
a specific integer size which may not be large enough to store a
pointer.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: Icfe219e1bbb2d06b3ef05710fac5b7091d340251
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5142
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In my thought, this await_req will not have performance improvement,
and it makes the code more complicated, so remove it.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I97d86448fc273098d001a08d914ad45ddac07e16
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5149
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
As part of nvmf_transport_qpair_fini, FC transport needs to cleanup
all the resources used by that QPair on the shared hardware. This
hardware cleanup is asynchronous in nature.
FC transport code to use this functionality will be pushed shortly.
Change-Id: I5606a33dff45971badd74e0cc087b132b56af076
Signed-off-by: Naresh Gottumukkala <raju.gottumukkala@broadcom.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5100
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
There are many duplicated codes about sending response for writing bool result.
That we need a function to do this.
Then we can reduce many codes.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic439111b1e9ca1013f8c657ab925f0c27a7be699
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5033
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When ANA is enabled, if a new namespace is hot-added to the subsystem on
the target side, the Linux host will expect to receive an ANA change
notification after the namespace change one. Without the knowledge of
the namespace's ANA state, the host will not register the associated
block device, so users cannot see the nvme device.
In this patch, ANA change is notified right after namespace change, so
host will get ANA log, update the namespace's ANA state and then
register the block device if it is alive.
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: I2faf61d344eb6de7efb2fde3cb1013cf0efd3c2e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5097
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reference the struct members only when it is necessary.
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I44208dfde20f74ab96e494445a464bf45f7ab408
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5070
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Remove linkbreak related code that is not being used anywhere.
Signed-off-by: Naresh Gottumukkala <raju.gottumukkala@broadcom.com>
Change-Id: I8ab2a8337fd5615bdb75744a7abcd28ec3d41f2b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4977
Community-CI: Broadcom CI
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added asynchronous event notices for discovery log change
as per nvme fabrics spec 1.1. This allows a host with persistent
connection to discovery controller to automatically connect to any
new subsystem available to the host automatically.
According to nvme fabrics spec 1.1, if the connect command specifies a
non-zero keep alive timer value and the discovery controller does not
support asynchronous events then we need to return Connect Invalid.
Since SPDK does not implement this check instead added support
for asynchronous events in discovery controller.
Change-Id: I4cade5f7d24826ce97a2fa2b4ca688a1d728c1db
Signed-off-by: Madhu Adav MJ <madhu.adav@flipkart.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
When user used CUSTOM transport, follow this step can reproduce:
1. run ./nvmf_tgt
2. ./rpc.py nvmf_create_transport -t ABC (ABC is the transport name)
3. ./rpc.py save_config >> config.json
Then in config.json :
{
"subsystem": "nvmf",
...
"config": [
{
"method": "nvmf_create_transport",
"params": {
"trtype": "CUSTOM",
...
}
]
}
trtype should be ABC , because nvmf_create_transport need pass
the transport name to create transport.
Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Change-Id: Iaf24837b649a1736568902f898d48135dac0882d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4973
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Fixes issue #1635.
Under rare circumstances, the CC.en and CC.shn are both set
which then results in setting the association timer twice.
This scenario was observed during hot plug testing when the
initiator tries to reset the subsystem that contains the
removed device.
The end result is that when the ctrlr is destructed, then
one of the timers can still fire and access freed memory.
Change-Id: Ie5880ab325a28f19361f73712bdeb5b58894ee68
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4935
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Issue:
ctrlr.c:1851:13: warning: Although the value stored
to 'copy_len' is used in the enclosing expression,
the value is never actually read from 'copy_len'
next_pos = copy_len = 0;
^ ~
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ie6d402cd4ad206ae5713deb15e2d03929bac94f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4823
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Move data used in IO path to the beginning of the structure,
eliminate several holes
Change-Id: I45202f31c888fba32307a8b8ae2b62e7e601a32f
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If the user decided to disable ICD then we have several side effects:
1. SPDK prints several warnings/errors
2. SPDK doesn't create recv pipe and doesn't set SO_RCVBUF socket option.
I think that we should not rely on ICD only when we create recv pipe or
set SO_RCVBUF since data may be transferred in sgls via R2T/H2C and
we still need recv_pipe and SO_RCVBUF for better performance.
Alternative option is to set recv_buf_size as a maximum between
ICD and io_unit_size
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ida71ecc099f9a9355e4617f13315a341872d1cb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This parameter represents the number of control messages to be
allocated per poll group, specific for TCP transport.
The new parameter can't be zero.
Change-Id: I8ae198c0b46e9a5850a80492aa6260f0c6ef885e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
According to the SPEC we should support up to 8192 bytes
of ICD for admin and fabric commands. Transport configuration
parameter in_capsule_data_size is applied to all qpair types -
admin and IO. Also we allocate resources when we get a connection
request, so we don't know qpair type at this moment.
Create a list of buffer in TCP poll group to support ICD up
to 8192 bytes when configuration ICD is less than this value.
The number of elements in this pool is hardcoded, it is planned
to add a new configuration parameter later.
Fixes issue #1569
Change-Id: I8589e3e2ea95d515f5503c6de7c1ee40aaf7b6da
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4754
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If the initiator sends a property set command to a valid register, but
the value contains bits that are reserved by the NVMe specification,
don't fail the whole command. Just log an error message.
Previously, any valid bits set would take effect but then the command
would also fail. That confuses the initiator and thinks it must retry
the full property set.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I566bc68c4469b9f41c69902e276f825c86683075
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
If a host matching the removed hosts' NQN exists, it is now
disconnected.
Change-Id: I0bec29eda2dc220114b9197d4eb765899b9e1517
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4684
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
- Removed slew of conf.h includes
- No longer require mk vars that include conf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica7e8e8bf1d4a5d0b0200bfe689aa13afd77bfaf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are more transport on the way and we don't want to add
all their various opts into the single, generic structure.
We'll pass the JSON structure to transports instead. Then
the transport code can custom pull from the JSON any param
it wants.
To complement that, transports will now also have their own
JSON config dump callback. This was only done in the generic
nvmf.c so far, with conditions for RDMA and TCP.
Change-Id: I33115a8d56cec829b1c51311a318e0333cc64920
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
For example:
Got JSON-RPC error response
response:
{
"code": -32602,
"message": "Invalid transport type 'rdma'\n"
}
The \n here is redundant.
Change-Id: I30a22f93f2be2550fdbe2af2d90eaa1c381dc7ae
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This will disconnect all connections to a subsystem from a given
host identified by HOSTNQN.
Change-Id: Ibc9cea1f08a58a05dbac3a0bb47df8d8a58e7c10
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4556
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
There is nothing left here, so remove it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib947d42bc577dbebb4650b1be885e05a80f8f8cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Add an new API spdk_nvmf_subsystem_add_ns_ext() to pass not bdev but
bdev_name to fix the race condition due to the time gap between
spdk_bdev_get_by_name() and spdk_bdev_open(). A pointer to a bdev is
valid only while the bdev is opened.
spdk_bdev_open() has been replaced by spdk_bdev_open_ext() but the
issue still existed.
Update the corresponding unit tests accordingly.
Then replace the internal of spdk_nvmf_subsystem_add_ns() by
spdk_nvmf_subsystem_add_ns_ext() call.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifcaa2121129ef22d5e61c9a8f7c640ff37a64485
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4485
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
There are operations on nvmf which depends on proper values of qpair
attributes which can be intepreted as internal state.
e.g.
nvmf_ctrlr_process_fabrics_cmd execution relies on qpair->ctrlr
spdk_nvmf_qpair_disconnect relies on qpair->disconnect_started
As poll group add is like a registration of qpair into nvmf lets try
to initialize it to a defined and expected state.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I10494e7f70ff58ec5460cab1de8a52fd21cc4a48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch removes the string from register component.
Removed are all instances in libs or hardcoded in apps.
Starting with this patch literal passed to register,
serves as name for the flag.
All instances of SPDK_LOG_* were replaced with just *
in lowercase.
No actual name change for flags occur in this patch.
Affected are SPDK_LOG_REGISTER_COMPONENT() and
SPDK_*LOG() macros.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I002b232fde57ecf9c6777726b181fc0341f1bb17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
a pause
This now also takes a lock instead of requiring a pause of the whole
subsystem.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I7de174f3f56d2b3767e723387c4f2257107d8b19
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4581
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The list of allowed hosts is only checked during handling of CONNECT
commands - not in the main I/O path. Protect that list with a mutex
instead of requiring a full pause of the subsystem to allow
dynamic management of the allowed hosts without impacting any
active I/O.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I3f7e87cc1fa6de200c422928c07153fc60fab28c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Pack all of the hot data into the first cache line. The first cache line
covers everything up to and including the ctrlrs TAILQ.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I184520661743aec91b3bb3d81e53fe8610c9383e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This saves 2 bytes and allows it to pack nicely with the
changing state bool (which must remember separate for atomic
operations).
Change-Id: Ibb92ae3c74306e60385ae23d0aaf877f33a69095
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4553
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Without this change nvmf_ctrlr_create() will fail to lookup
the subsystem listener matching this qpair.
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I855baa16e996737b60dbd745ce84f8c0bc024cf1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
LOCAL_SYS_LIBS is meant to define *direct* system
library dependencies for a given library. libuuid
is directly used by the SPDK util library and then
other SPDK libraries use uuid indirectly through
util.
So only the util library should include uuid in
LOCAL_SYS_LIBS.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia0d2d63f48e6f89891164cf2f9dc4c7a6476d4e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4366
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
It should be 16 but not 6. For example, it will have 16 priorities
when configuring ADQ with Intel's 100G NIC.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Iebdf7b379c15f3b5fd16dba2ad87ec55af04577f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
In the both normal and exceptional case, the mutex
will need to be destroyed.
Change-Id: I39c815f2adffbd3786b45a938c476dcbb66a438f
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4339
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: JinYu <jin.yu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Unlike ADMIN and IO commands, the FABRIC command is only processed
in the ctrlr.c file.
Change-Id: Ic4e01c7f81c98631a2c7cb603343b301f8ba63e1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4307
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The poller is now created internally to the library whenever a target
is constructed. Applications are not expected to poll for connections
any longer.
Change-Id: I523eb6adcc042c1ba2ed41b1cb41256b8bf63772
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Remove some of the boilerplate code from each case and
replace with just an spdk_msg_fn assignment.
This also reduces the size of an upcoming change needed
in this function.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia209073cfb66032f2cca6bb44a09e1984ef2110c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4257
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In C language, we cannot use constant at compile time. Hence the
local array _ana_desc[] is not a fixed size array but a variable
length array.
We can avoid using variable length array by changing const variable
to macro constant.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7333a8078d3102c4bd5088f56f6530846854c85f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4093
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Add an new RPC, nvmf_subsystem_listener_set_ana_state.
Find the specified subsystem listener, and then set the ANA state
of the listener by calling nvmf_subsystem_listener_set_ana_state().
By adding a string and an enum to the existing context structure,
nvmf_rpc_listener_ctx, and adding an operation type to the existng
enum, nvmf_rpc_listen_op, reuse the existing code and data as much
as possible.
Besides, insert line break into a few long lines and fix wrong
error log.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6fb2dfbb1f9c5f56848eba21d2a733fbed802614
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an internal API nvmf_subsystem_set_ana_state() to change the
ANA state of the subsystem listener whose trid matches.
ANA optimized state, ANA non-optimized state, and ANA inaccessible
state are supported. ANA change state is not used and ANA persistent
loss state is not supported.
After changing the ANA state of the subsystem listener, on each poll
group, controllers, whose the subsystem listener match, send ANA
change notice.
Initiators query ANA log page anyway if they receive ANA change
notification. False positive notification should be avoided but is
acceptable.
To avoid any concurrency conflict, simply compare ctrlr->listener and
the passed listener.
It may be better to execute nvmf_subsystem_set_ana_state() on the
subsystem thread but currently the RPC thread adds and removes a
listener to and from the subsystem, respectively, and the subsystem
has been suspended while executing nvmf_subsystem_set_ana_state().
Hence we keep this as a future enhancement.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If1910b79dd33d904114e258ae2c5e868947cdc52
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4079
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
If the ANA reporting feature is enabled for the subsystem,
- set ANA Change Notice of Asynchronous Event Configuration to 1
- set ANA Change Notice of Optional Asynchronus Event Supported to 1
- set ANA Non-Optimized state and ANA Inaccessible state of ANA
Capability to 1.
ANA Change state is not used and ANA Persistent Loss state is not
supported for now.
The next patch will actually support ANA Change Notice using an new
RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4db2e33dd2879cdf995adcab41ef53728b27a201
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If we are already in the desired state,
just call the callback directly from the
subsystem_state_change function. That way
we save a lot of message passing.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6cf8563524610d9125d53266e3c0e179e064bf63
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3760
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is important to avoid doubling up on state changes
and hitting asserts.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If8797ea13a5c224cee85e53e9b2542012423b37f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3759
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
There is no need for this interface to be async.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I1f21b53e90b7d165b6b5fb2e1226ce7591966b58
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4181
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
It was introduced for the purpose of executing fabric cmds when
subsystem and qpairs are not active. It was rather workaround than
solution for transport type like vfio-user. spdk_nvmf_request_exec
is a preferred way of passing request obj into nvmf layer.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I4f989de27bfd494c744017599909c2e200f0f233
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4180
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Add an new RPC, nvmf_subsystem_get_listeners.
ANA state is per listener and per subsystem, and is stored in
subsystem listener. We can return ANA state by the existing
nvmf_get_subsystems RPC but it's confusing that listen addresses
have ANA states.
To change ANA state, we will provide a RPC to change ANA state of
only one selected subsystem listener.
To query ANA state, it will be convenient to get ANA states of all
listeners of one selected subsystem.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic3baad6eac65d7af6e0cab2c4059e1458d41e6e2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Data structure and macro constants for multiple listen addresses
and namespaces are not used anywhere in nvmf_rpc.c
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idd8bc61e22f9e9918a88f017a024cab239ff5e53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4060
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an new RPC, nvmf_subsystem_get_qpairs to retrieve the list of
qpairs of an NVMe-oF subsystem.
This RPC will be usable to verify if NVMe ANA works.
Pause and resume the subsystem to access the qpairs safely.
One subtle issue remains. The JSON RPC returns success even if
resuming the subsystem fails. Write FIXME to address this.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9d90a01b1117dee00d85b2e21b4f4d02d80db531
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4050
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Usage of spdk_thread_get_count is wrong since there might be many
threads allocated by other modules. Transport buffers are used by
transport poll groups, their number is equal to the number of cores.
Change-Id: I4bc748e93c3b204bf3b3ec73f17257b927a7f428
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3882
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When we try to evenly divide transport buffers between poll grouos,
e.g. when we run spdk_tgt on 8 cores, set num_shared_buffers=32768
and pg buf_cache_size=4096, the last pg can't retrieve enough
buffers to fill cache. In my case if only got 4040 buffers out of
4096. Missing 56 buffers were cached by previous poll groups.
That occurred due to mempool has per lcore cache of 512 elements
and when it becomes empty, the cache is refilled. It seems that
each poll group cached extra 8 buffers.
The issue doesn't occur when we use mempool_get_bulk.
Change-Id: I866d58aa03986a3cffe27402b12f9a2519097f83
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3881
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Factor out the internal of rpc_nvmf_subsystem_get_controllers() into
a function rpc_nvmf_subsystem_query() to use it for the upcoming RPC,
nvmf_subsystem_get_qpairs.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibe62bcfadf6b33ef26c018a3667f280b6fcd8fdf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4049
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
For nsid, use SPDK_NVME_GLOBAL_NS_TAG rather than raw number
0xffffffff wherever possible.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I23e989786263172e13bab40c011cf58beb06fabf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4055
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This can happen and we should make a best effort to return
the subsystem to a coherent state when it does.
maybe fixes: issue #1416
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic3d0376984733e6664295305be82fca678c515b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3437
Community-CI: Broadcom CI
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This can happen and we should be prepared for it.
Maybe fixes: issue #1416
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I77f48dbcabf702f88df56ad7e866bbcb830fc239
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3393
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
For I/O commands, block them if ANA state is inaccessible, persistent
loss, or change.
For Identify command, clear capacity field (nuse) to 0 if ANA state
is inaccessible or persistent loss.
For Get Features command, block features, error recovery, write
atomicity normal, reservation notification mask, and reservation
persistence if ANA state is inaccessible, persistent loss, or change.
For Get Log Page command, error information page does not return
any data yet, and hence there is no change.
For Set Features command, if ANA state is inaccessible or change,
block the command if NSID is 0xFFFFFFFF or if feature is error recovery,
write atomicity normal, reservation notification mask, or reservation
persistence, or if ANA state is persistent loss, block the command.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I15dd593227e451aa2247c53da42b6acad1757907
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4043
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Add ANA state to struct spdk_nvmf_subsystem_listener and initialize
it to optimized.
Then ctrlr->listener->ana_state is referred when creating ANA log page.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I978424e51d3f23ca72dee30192bc2693abfe203d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4012
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We will have ANA state per listener and per subsystem. On the other hand,
NVMe specification defines ANA state per controller.
However, it is possible that I/O qpair and admin qpair are different
listeners on a single controller.
Let's check if I/O qpair is on the same listener as admin qpair if
ANA reporting is enabled.
The case that I/O qpair is on a different listener from admin qpair
is not usual and so the purpose of this check is just to guard SPDK
from any unexpected behavior.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idb8d255de7f998e45a59a120c2ed5803258873f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4026
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Find the subsystem listener whose trid matches req->port->trid when
creating a controller, and store it in the controller.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iea343b8d8ae827b554df2245b67aed113469c592
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Add trid to struct spdk_nvmf_qpair and initialize it at initialization.
admin_qpair->trid will be used to get the corresponding
subsystem_listener via nvmf_subsystem_find_listener() and add it to
struct spdk_nvmf_ctrlr in the next patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0d1a41aede60de88747eff16c7e04f63d0702596
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4009
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The new function () will be used in the following patches.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I788cfb38d75c3f1f64e1754912b776a80f0f1be8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4007
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We stopped the poller to early, so we were not able to
reap all completions on ibv CQ, so RDMA qpair was not freed.
This patch stops the poller when all references to poll group
are released (all qpairs are destroyed)
Fixes#1578
Change-Id: I15c1697db13aef9da7567c7312476306c3ee1d62
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3962
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
ANA transition time shall be non-zero if controller supports ANA
reporting. Linux NVMe host sets this value to 10, and we don't
have any reason to change from that.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I61396695dacf47fad40e3cea3311e555729d9e3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3909
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This will be used in another place later.
This patch is part of a series aimed at improving recovery
when we are fail to change the subsystem state.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I24bfbeb3d006584003164540d6ede540dbcafa86
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3392
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Failed qpair will be destroyed on generic nvmf layer during handling
of error code returned from spdk_nvmf_poll_group_add.
The current approach leads to heap-use-after-free.
Change-Id: I99331150fa36a3c3c18176589afb973dee449b3a
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3538
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Currently rdma acceptor handles only one ibv event per poll
Taking into account the default acceptor poll rate (10ms), it can
take a long time to handle e.g. LAST_WQE_REACHED events when we
close huge amount of qpairs at the same time.
This patch allows to handle up to 32 ibv events per acceptor poll.
Change-Id: Ic2884dfc5b54c6aec0655aaa547b491a9934a386
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3821
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
SPDK poller uses microsecond as the input parameter, so we need to
change the correct value when opts.association_timeout is expressed
by millisecond.
Change-Id: Ia674f0115ea176b998e4c0c70b8ce75b28984701
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3861
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
After supporting ANA reporting by default, Linux kernel 5.3 reported
error when parsing NVMe ANA log. The newer kernel fixed the issue
but we should optionalize ANA reporting feature to avoid error for
Linux kernel 5.3 or before.
Add a bool variable ana_reporting to struct spdk_nvmf_subsystem
and disable ANA reporting and initialization of related variables
if it is false. We can expose MNAN (Maximum Number of Allowed
Namespaces) even if ANA reporting is disabled. But MNAN is not
required if ANA reporting is disabled. So do not set MNAN if it is
false too.
Add a public API spdk_nvmf_subsystem_set_ana_reporting() to set
ana_reporting by the nvmf_create_subssytem RPC.
The next patch will add ana_reporting to nvmf_create_subsystem RPC.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icc77773b4c9513daba2f1a9fdaf951d80574f379
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3850
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an new RPC, nvmf_subsystem_get_controllers to retrieve the list
of NVMe-oF controllers of an NVMe-oF subsystem.
One of the main use cases will be to get identification information
of NVMe-oF controllers to configure their ANA states dynamically.
Pause and resume the subsystem to access the controllers safely.
One subtle issue remains. The JSON RPC returns success even if
resuming the subsystem fails. Write FIXME explicitly to address this.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibf8d1cf56850a705e343b86022d101b4c7204199
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3848
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
A subsystem RPC is not transitioned to a paused state when there
are ios outstanding (tracked by subsystem poll group).
In general AERs, are not tracked as outstanding IOs. However,
there are 3 paths in nvmf_ctrlr_async_event_request which do not
adjust the outstanding io count.
If we get into any of these 3 paths, the subsystem pause can hang
forever.
The issue was reproduced with hot plug stress testing under load.
We can get into the second path (SPDK_NVME_ASYNC_EVENT_TYPE_NOTICE)
under these circumstances:
- An AER completion is sent to the initiator due to a namespace change
(e.g. hot remove/add)
- In this case, type is set to SPDK_NVME_ASYNC_EVENT_TYPE_NOTICE
- The initiator sends a new AER admin command, hitting the second path
where we return without adjusting the outstanding ios.
Fixes: 1552
Change-Id: I45f781966cc1e9a601b2305c7985a21154d802e8
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3854
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: JinYu <jin.yu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Generally, this patch did the following work:
Remove the destruct poller. I think that we do not need this,
the destruct poller is specially for Softwaare RoCE case.
Since SoftRoCE will not have IBV_EVENT_QP_LAST_WQE_REACHED event,
we will not wait the last_wqe_reached flag when srq is enabled.
So we can avoid using the poller.
And the purpose of this patch is to solve the coredump issue.
For example, if we run rdma local test such as, e.g.,
test/nvmf/host/bdevperf.sh --transport=rdma
The coredump reason: the qpair is freed twice. Because for RDMA transport,
we do not really remove the qpair from the group if the upper layer
does it.
The first time is called by nvmf_rdma_destroy_drained_qpair in nvmf_rdma_poller_poll,
and the second time is called by nvmf_rdma_qpair_reject_connection in
in nvme_rdma_close_qpair. Since nvme_rdma_close_qpair will always called,
so we need make sure that the qpair will be close after calling this function.
Otherwise we will have the double free qpair. So our approach here is add a flag
("to_close")in rqpair structure and make sure the rqpair be freed after the
"to_close" is set nvme_rdma_close_qpair
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I6f97debbcd29bbb7c6e3f9725907b4102a1d2892
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3661
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Helps us avoid adding a new I/O qpair while the ctrlr
is being destroyed.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I3bf9318b075125b9d432b885fa9f6f2f44d422d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3686
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This data structure is not used.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I143fb9256f692d7bd9bb5e14cdc479f64ddcef45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3746
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
1. Retrieve actual IBV state when we receive WC with bad status
2. Don't log an error if WC status is IBV_WC_WR_FLUSH_ERR. This
means that we are performing qpair cleanup and this WC is expected.
Change-Id: Id23634092f537861e66ca0f83ab79db9e052507b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3736
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
From the time a shutdown is initiated the controller shall disable
Keep Alive timer.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Id499dabce1913b9da2f0b3fd961fdfc8b621afa9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3462
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
After CC.EN transitions to ‘0’ (due to shutdown or reset), the
association between the host and controller shall be preserved for at
least 2 minutes. After this time, the association may be removed if
the controller has not been re-enabled.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Change-Id: I4734600067fd4b7306b46f1325fdd5031e81c079
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2984
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This call can be made directly now that
spdk_nvmf_qpair_disconnect is thread safe. It's
actually better that we do it this way, because
the qp destruct call is guaranteed to block until
the ib events associated with it are acknowledged.
this means that by processing the disconnect before
we ack the event, we will have valid memory to do
the atomic checks.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If6882b7dc568fe4c35f4a35375769634326e9d76
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3681
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We should use this function as the synchronization point
for all qpair disconnects.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic685ac3481765190cc56eeec3ee24dad52e336c9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3675
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This function should be the synchronization point for all
disconnects regardless of whether they begin on the transport,
from an RPC, or in response to application termination.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If3553ab3a9e265b0938c84832cb9f774852d7565
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3674
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
SPDK NVMe-oF controller creates a ANA group for each namespace,
ANA group ID matches namespace ID, and default ANA state of ANA group
is optimized, and the MNAN field is set equal to the NN field.
If a ANA log page contains multiple ANA group descriptors, it has
one or more descriptors will not be 8 bytes aligned. Hence we create
one descriptor and copy it to the ANA log page at a time.
Change count will be supported later.
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I56ba6aa78983480caa3dfbf22aefc9aeabfd5405
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2920
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are 2 messages passed between when
_nvmf_ctrlr_free_from_qpair is executed and when
nvmf_ctrlr_destruct is executed. That leaves time
when the controller->qpair_mask is not a valid
pointer, but it is still in the subsystem
controllers list.
The purpose of this patch is to close that hole.
It is part of a larger series aimed at cleaning up
the controller destruct path.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I0c0199c8392ee278f36df56f599beb10e7a46948
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3685
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This API differs from spdk_nvmf_tranpsort_stop_listen in
that it also disconnects the qpairs associated with
that listener.
Change-Id: Iadfc6d2debc0ef8f1a8cd5db4f20168aeae8264d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3279
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
