Depending on the number of cores there are sporadic issues getting
elements of that pool although free elements are there during poll
group creation. Operation returns -ENOBUF. It results in odd notice
msg.
"nvmf_transport_poll_group_create: *NOTICE*: Unable to reserve the
full number of buffers for the pg buffer cache. Decrease the number of
cached buffers from 455 to 1366"
In this case 1366 is the actual number of available elements in the
pool. Few poll groups suceeds and few are ending up with the buffer
cache size set to 0.
Issue has been rootcaused as bug or behaviour change in DPDK v22.01.
Consider example:
We create DPDK mempool with 4K buffers, cache of 256. When first poll
group requests 512 buffers, DPDK mempool first looks in its per-core
cache, sees no buffers (mempool buffer cache doesn't get prepopulated)
and then requests 512 + 256 buffers from the backing pool. It returns
512 of the buffers to the user, and puts the other 256 buffers in the
cache ...it should only request 512 buffers total. For 8 cores and 512
buffers requested only 5 cores will get their buffers.
Disabling mempool cache seems to workaround the issue. More effective
cache is already implemented on nvmf generic layer.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I3149dea95a4f24a75dd0074eda9468c4856d901d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16913
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add error check to unclaim mechanism. Issue #2920
showed that unclaiming CPU locks might fail and
we should catch errors to determine the cause.
Change-Id: Ifdfb7db2595d73f8bae13418ef145ad80e1d07ef
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16958
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add implementation of uuid_generate_sha1() for systems
that do not have this function in their system libraries.
Use uuid_generate_sha1 from uuid.h inside a new function
spdk_uuid_generate_sha1(). The reason for this addition
is to prepare for UUID generation correction to conform
to standards.
First part of series addressing #2788.
Change-Id: Ib357aa1ee832e886288d176d8a47efdaa326f537
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16414
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Return the real spdk_fd_group object so it can later be nested.
Change-Id: I84c8a174c7d177799fa484b350269082c61b18a5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15474
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Don't assume spdk_thread_poll() will ever get called. Instead, send
a message to process the exit.
Change-Id: Idd98e7e8164c5efebd0d7c9287e62731e7cbc998
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15551
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Move away from relying on spdk_thread_poll() to do clean up in interrupt
mode. In the future, we don't want to have spdk_thread_poll() called at
all.
Change-Id: I5318a7889601a3d3463e35419918b7305f68ee8d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15550
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Eventually, we want to allow merging of spdk_fd_groups, removing a level
of indirection. That means that some interrupt handlers won't
necessarily fire with the spdk_thread context already set. Set it in the
wrappers to ensure it's right.
Change-Id: Ief18d58cf3ee005c2969a9c0ee132b34b24cbd61
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15476
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
This unifies the poller fds with the interrupt mechanism internally.
Change-Id: I57a270260981ff54670365dddb33a1d9bdb56781
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15754
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Added initialization of the prev_crc variable to avoid compilation error:
idxd.c: In function ?spdk_idxd_submit_copy_crc32c?:
idxd.c:1138:51: error: ?prev_crc? may be used uninitialized [-Werror=maybe-uninitialized]
1138 | desc->crc32c.addr = prev_crc;
| ~~~~~~~~~~~~~~~~~~^~~~~~~~~~
idxd.c:1081:18: note: ?prev_crc? was declared here
1081 | uint64_t prev_crc;
| ^~~~~~~~
Change-Id: I6b93d5d85b52e20f8a2c313c41b740f66eebe1c7
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16900
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Fix for number of dwords which is 0 based as per spec.
Use bitwise operators instead of division and modulus.
Change-Id: Ib315bf9394ef599317f41429742e7b8054069549
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
They aren't cleared before a task is submitted and might store pointers
from a previous operation. This can lead to issues if the previous
operation was using memory domains and we submit the task to a module
also supporting memory domains.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Icafb924c2e936ee6a83d921ae48e953b98f00841
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16848
Community-CI: Mellanox Build Bot
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Remove libuuid usage on FreeBSD and add dedicated implementation of
spdk_uuid API using functions from the standard library.
Fixes: #2878
Change-Id: Ie49ccb2842acad6064bffd789e4f64b7365b6e5c
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16558
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
An example of async operation which can be handled on specific
transport layer could be creation of spdk thread followed by
a poller registration.
This change also aligns with transport destroy which is already
async operation.
Current transport create function is marked deprecated and is meant
for transports supporting sync create only to maintain backward
compatibility. Async version supports both create operations.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I1f5a477819e58f30983d26f81a1416bed1279ecf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16463
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The spdk_bdev_ext_io_opts structure is used to pass extra options when
submitting a bdev IO request, without having to modify/add functions to
handle new options. Additionally, the structure has a size field to
allow adding new fields without breaking the ABI (and thus having to
bump up the major version of a library).
It is also a part of spdk_bdev_io and there are several reasons for
removing it from that structure:
1. The size field only makes sense in structures that are passed
through pointers. And spdk_bdev_ext_io_opts is indeed passed as a
pointer to spdk_bdev_{readv,writev}_blocks_ext(), however it is
also embedded in spdk_bdev_io (internal.ext_opts_copy), which is
also part of the API. It means that each time a new field is added
to spdk_bdev_ext_io_opts, the size of spdk_bdev_io will also
change, so we will need to bump the major version of libspdk_bdev
anyway, thus making spdk_bdev_ext_io_opts.size useless.
2. The size field also makes internal.ext_opts cumbersome to use, as
each time one of its fields is accessed, we need to check the size.
Currently the code doesn't do that, because all of the existing
spdk_bdev_ext_io_opts fields were present when this structure was
initially introduced, but we'd need to do check the size before
accessing any new fields.
3. spdk_bdev_ext_io_opts has a metadata field, while spdk_bdev_io
already has u.bdev.md_buf, which means that we store the same thing
in several different places in spdk_bdev_io (u.bdev.md_buf,
u.bdev.ext_opts->metadata, internal.ext_opts->metadata).
Therefore, this patch removes all references to spdk_bdev_ext_io_opts
from spdk_bdev_io and replaces them with fields (memory_domain,
memory_domain_ctx) that were missing in spdk_bdev_io. Unfortunately,
this change breaks the API and requires changes in bdev modules that
supported spdk_bdev_io.u.bdev.ext_opts.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I49b7524eb84d1d4d7f12b7ab025fec36da1ee01f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
TP4146 introduced support for two new IO commands,
IO management receive and send.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Change-Id: Iaf37310b84e278df043dcf71a0c2ef912c2fca8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16520
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
TP4146 added support for 4 new log pages.
These are FDP configurations, reclaim unit handle usage,
FDP statistics and FDP events.
Updated the identify example file accordingly.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Change-Id: I5a20b728605257774d72bc184b50bc5008e142ea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16518
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Format LBA size (FLBAS) is updated to have:
Bit 3:0 as least significant 4 bits for format index
Bit 6:5 as most significant 2 bits for format index
NVMe format command fields are updated accordingly.
Add a new helper function to fetch the correct format index.
Update examples and unit test files accordingly.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Change-Id: I2d6d9045b9d65ae91cb18843ca75b59cc27ed2f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16515
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
It isn't used in this function and the callers always pass NULL.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I07baa13a25b1e4e0b8832a093a53250392b10f10
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16682
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This should help catching bugs when a failed sequence gets cleared its
failed state.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I9389a2610e94e766aaf4185445c36442c4d4a1f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16545
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This will allow a platform driver to allocate a buffer in case it cannot
execute the whole sequence and the destination buffer of the last
operation is a "virtual" accel buffer.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia947cf553619828a170c5d0563b4c355d7b5ead5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16377
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This will allow drivers to check if a task is using buffers from accel
domain. This is just a helper, since the same can be achieved by
calling `spdk_memory_domain_get_first("SPDK_ACCEL_DMA_DEVICE")`, but
there's only a single accel domain and it is a bit special, so it makes
sense to have a dedicated helper function for getting it.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I07db7445ed9b109e66ecdbc0483a6a158a551070
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16376
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The goal of a platform driver is to execute chained accel operations in
the most efficient way possible. A driver is aware of the hardware
available on a platform and can execute several operations as a single
one. For instance, if we want to do DMA and then encrypt the data, the
driver can do both at the same time, if the hardware is capable of doing
that.
Platform drivers aren't required to support all operations. If a given
operation cannot be executed, the driver should notify accel to continue
processing a sequence, via spdk_accel_sequence_continue(), and that
operation will processed by a module assigned to its opcode.
It is required however, that all platform drivers support memory
domains, including the "virtual" accel domain. A method for allocating
those buffers will be added in the following patches.
This patch only adds methods to register and select platorm drivers, but
doesn't change the way a sequnce is executed (i.e. it doesn't use the
driver to execute it).
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97a0b07e264601ab3cf980735319fe8cea54d38e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16375
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If req->data is set, with all the previous changes, then req->iovcnt
should also be more than zero.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I29b5f45541c9dba2dd896109dd43d2b5321ec467
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16274
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Also deprecate the existing spdk_nvmf_request_data() API, which is
incompatible with iovecs.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I44df8ff30a431873a0c2f34b0cdb58df858fd7e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16200
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Use req->iov instead of req->data in reservation handling code.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I6d79711d03f45bd5e118c6324d22decad887a788
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16199
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If spdk_sock_flush() returns an error, there's no reason not to
disconnect the qpair, as it usually means that that socket's connection
has been terminated.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I54e9bebc38e2a24a3baf69eb18ec3c654b210318
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
The bahavior of spdk_sock_flush() was changed in 5433004ec to return the
number of flushed bytes and -1 with errno set to EAGAIN in case nothing
has been flushed (instead of returning 0). Therefore, we shouldn't
treat EAGAIN as an error in nvme_tcp_qpair_process_completions().
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5473488b5b408cdc739921046f1a0cc2c98f98de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16643
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This never happens, as requests in this state are always immediately
transitioned to other states.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0408ed9d8003d364bc38c86a9a50312721ab1284
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16642
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It is possible for requests waiting for R2T ACK to receive H2C PDU
before receiving the ACK. Therefore, the following sequence:
1. Host sends a write request to the target.
2. Target sends R2T PDU to the host and sets request's state to
AWAITING_R2T_ACK.
3. Host sends H2C PDU to the target, but it doesn't reach the target
yet.
3. Host sends an abort command to abort that request. Request's state
is changed to READY_TO_COMPLETE.
4. Target receives the H2C PDU, sees that request's state is
READY_TO_COMPLETE, which is unexpected, and terminates the
connection.
will cause the target to terminate the connection, which is obviously
incorrect.
So, to avoid that, we can treat AWAITING_R2T_ACK state in the same way
as TRANSFERRING_HOST_TO_CONTROLLER and register a poller waiting for the
state to be changed.
Fixes#2789.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idddc627050000b74663dba397dc14d10aa0e284f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16641
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
We don't need to allocate 2MiB aligned memory address for
vrings, this will waste memory and may invoke dynamic
memory allocation in DPDK sometimes.
Fix issue #2846.
Change-Id: I6410d417f92623b44c375359d5e2b5ec8ed815c0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16651
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Starting in SPDK 23.01, calling spdk_bdev_register() and
spdk_bdev_examine() from a thread other than the app thread was
deprecated. This commit removes the deprecation and as such calling
these functions from a thread other than the app thread is an error.
As a side effect of this commit, all bdev module examine_config() and
examine_disk() callbacks will be called on the app thread.
Change-Id: Idaae06608101e2a513d9312ac5544ffe94effe4a
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15826
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
If multiple claims exist on a bdev, examine_disk() is called for each of
them.
Change-Id: I0a6dc3e4bd1da20bbcbddf97a16e04c62c82354c
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15290
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
This commit has no functional change. It refactors an if statement into
a case statement in preparation for supporting claims v2.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I1862428c91a7066ad9079878d4c1b690a5ef631c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15289
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This implements the v2 claims API. Compared to the original v1 claims,
v2 claims:
- Support read-write-once, read-write-many, and read-only-many claims.
- Are claimed with spdk_bdev_module_claim_desc().
- Are associated with a bdev descriptor that is passed to
spdk_bdev_module_claim_bdev_desc().
- Are released upon close of the bdev descriptor used to obain the
claim.
- Cannot be taken when a descriptor other than the one passed to
spdk_bdev_module_claim_bdev_desc() has write access.
Later commits in this series are needed to fully integrate them with the
bdev subsystem.
Change-Id: I39a356f5893aa45ac346623ec9ce0ec659b38975
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15288
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
As new claim types are introduced, printing error messages about who
holds a claim will get more complicated. This refactors the error
message code into a function to prevent code duplication.
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Icdc5332214f3974e75baf11ba5ea02172c4275e1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15287
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Merge function _ublk_try_close_dev() directly into
function ublk_try_close_dev.
Change-Id: Ib4df28f1d69f56d72e3eeaeb2220eb15f5c37440
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16417
Reviewed-by: <yifan.bian@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
bdev is not proper to describe ublk devices
Change-Id: Ic6784b3062b770dce1c77dc409ba1ee3704bdef2
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16418
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
We need to use SQPOLL on the ctrl ring for now, due to
a bug in kernels <= 6.1. This ring is used very
infrequently, so the workaround has no real effect
on performance.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia5e65a6edb7b1b6c4c945ebda8941f98c2bcdb07
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16620
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
crc32.c uses SPDK_HAVE_ARM_CRC to use __crc32
intrinsics when available, but it isn't including
anything that would actually define SPDK_HAVE_ARM_CRC.
Only crc32c.c tried to define this previously, that
all got moved to crc_internal.h now, so we can just
include that here.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic5c2e6908508e32d2f3784304c5b58d3acd4c965
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16688
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The preprocessor code will be needed in another
crc-related .c file in an upcoming patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icc5e6f0d81f76093dfc973a9c1fe56271baf1ed8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16687
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
For JSON RPC, boolean response with false value may not be regarded as error.
Previously many cases were replaced to use
spdk_jsonrpc_send_error_response() explicitly. Replace one of the remaining
cases in this patch.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I1b4ffe015c2b2e28d411faf7763c2baca81f66f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16623
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For JSON RPC, boolean response with false value may not be regarded as error.
Previously many cases were replaced to use
spdk_jsonrpc_send_error_response() explicitly. Replace one of the remaining
cases in this patch.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ibfcb8c53662238b7ba8f08ecd1678953af8dc202
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16622
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The next patch will improve media mgmt notifications but it will be
almost same as _resize_notify() and _remove_notify().
On the other hand, there are a few differences between _resize_notify()
and _remove_notify(). _remove_notify() will be better.
To avoid duplication, unify _resize_notify() and _remove_notify() by
adding abstraction event_notify() and _event_notify().
Add unit tests for the complex race conditions.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibe2478479c61459c0da0db8d28c7273f05275e0f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16577
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Main core can be different than first core (default behavior) as it
can be specified by application argument. It can be useful to
determine if given thread is matching main core.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I25292a91ad677806eaf19ad68acdda0f28da6cfb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
These routines can only handle a single buffer; double check that is the
case, and fail if not.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I136482c27c73655887c49405f747b8ed073f7b69
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Use req->iov as needed, to make it easier to remove req->data later.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ie625f374e846f7e6afd6a5d143a5174d27d419b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16256
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Use req->iov as needed, to make it easier to remove req->data later.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I4095e3c4089b730db123705d0168cd409375cc43
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16196
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use req->iov as needed, to make it easier to remove req->data later.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Id23c4ef8018d6a7aad42c3d5054fa9addcf16f0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16195
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Refer to req->iov instead of req->data. As the queue connection code
already presumes a single data buffer, add some sanity checking for
this.
We also need to fix vfio_user.c as a result to correctly set ->iovcnt.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ib1e4ef3885200ffc5194f00b4e3fe20ab1934fd7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a new API for incremental copying in or out of an iovec, and replace
current code to use the new API.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I088b784aef821310699478989e61411952066c18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16193
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
During secondary process shutdown, when nvme device
would get detached, it would trigger env_dpdk to
send rte_eal_hotplug_remove event for the corresponding
BDF. But this isn't valid from a secondary process -
we can only attach/detach from the primary process.
Usually this would just result in a bunch of annoying
print messages as the secondary and primary process
sent rte messages between each other. But occasionally
one of the response messages from the primary process
could arrive just as the secondary process was going
through rte_eal_cleanup. The message would get
kicked to the DPDK intr thread, we would do bus_cleanup
which frees all of the PCI state, and then the message
would execute on the intr thread causing seg faults,
use-after-free or some other violation.
It's possible some kind of cleanup should be
implemented in DPDK, but for now, let's just not
induce incorrect behavior from SPDK, and don't send
the hotplug_remove messages from a secondary
process.
Fixes issue #2651.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie431a1f8e74503e1de1be36cbb9589682d6dc94a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16553
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We should always called the unregister callback on
the same thread that spdk_bdev_unregister() was
originally called. So save the thread pointer and
use an spdk_thread_send_msg() to make sure it gets
called on the correct thread when the unregister
finishes.
Also add unit test that reproduces the original
issue.
Fixes issue #2883.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib3d89368aa358bc7a8db46a8a8cb6339340469d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16554
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Since ublk_start_disk internally runs ublk_ctrl_cmd
asynchorously, the result should be returned after the
whole process of ublk_start_disk is completed.
Add a callback into ublk_start_disk parameter, and change
rpc_ublk_start_disk to send response in callback.
Change-Id: Icc0d9e8cb81f2b67bf99fdead423bfe8159714bc
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16500
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Use ublk_close_dev_done to release obtained resource
after UBLK_CMD_ADD_DEV is executed in the ublk device
start process.
Rename ublk_delete_dev to ublk_free_dev,
and ublk_close_dev_done to ublk_delete_dev
Also using the correct way -- io_uring_queue_exit()
to release uring in ublk instead of just closing uring fd.
Change-Id: I46a1ddea162adeb6dfe8c9b45082ed2f93f4b309
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16460
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Obtained resource can be directly released in error
handling of ublk_start_disk.
Fix the improper usage of _ublk_try_close_dev to release
resource in ublk_start_disk.
Change-Id: I291bbfb179b9df4deb8f3d346b2d660e6d17fdc1
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16459
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add function named ublk_ios_fini() to release memories
allocated in ublk_ios_init().
Fix the incorrect memory release for the failure inside
ublk_ios_init() by ublk_ios_fini().
Also move the memory release of ublk_io into function
ublk_delete_dev() instead of ublk_close_dev_done(). This
will be used by next patch.
Change-Id: I5c3fc31a4d7114e17fb86fc6facc8cccea27d6e7
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16442
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This will help with debugging more complex
ublk configurations - needed especially knowing
that ublk kernel driver is still a bit flaky.
Create a new LOG flag "ublk_io" for the existing
per-IO debug logs, and use the existing "ublk"
flag for ctrl-related debug logs.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic019c1e837b04dbf5d210c46a98cfbed732278a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I55a5a5111aa334807e22c9c622e33c69f0405a39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Sort the case statements by the normal order in which
the commands are submitted.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I203a70045cd48d30d1229c754ddb57e7e31460af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16455
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Instead of having the poller check the last
ctrl_cmd_op to determine which function to call,
just have ublk_ctrl_cmd store that function pointer
in the ublk itself. This keeps all of the
cmd_op-specific logic in one place rather than
splitting it between the submit and poller
functions.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I59f333d38a0d89cc4c50082177ff818135bcad37
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Some functions such as ublk_start_kernel,
ublk_stop_kernel and _ublk_start_disk were only
calling ublk_ctrl_cmd. Just call ublk_ctrl_cmd
directly and avoid the extra function indirections.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic7d211b9b730af816f7747e031bdaef865ece433
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
We don't need this parameter - it is simpler to
just determine the argument data (if any) in the
function itself based on the opcode.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8df8f9ec569c10ebb565efc7268fba1d50bcdf5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Every queue should be closed based on only its own
I/O information, not the whole device.
Remove function ublk_is_ready_to_stop().
Change-Id: Iec5a260f68d6f84c0af4c8e0b5380272049b1f5d
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
count was unused in release builds. Otherwise following
error is produced on clang build:
19:30:56 ublk.c:974:14: error: variable 'count' set but not used
[-Werror,-Wunused-but-set-variable]
19:30:56 int rc = 0, count = 0, tag;
19:30:56 ^
19:30:56 1 error generated.
Change-Id: If7ca88de37ed6e40826e09b055355c07f67c8869
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
This is a workaround but is necessary to fix the github issue #2874.
Due to some unknown reason, in nightly test with Intel e810 NICs
when a qpair is created with synchronous mode and connection errors
are detected, the qpair is destroyed even if requests for the qpair are
still inflight. Then, nvme_rdma_process_recv_completion() causes NULL
pointer acccess. To fix this NULL pointer access, change
nvme_rdma_process_recv_completion() to return immediately if rsp->rqpair
is NULL. Add a TODO comment to find a root cause and really fix the
issue.
One of the fixes for the issue #2874.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ic810922f7ea1b32373b15f4e0cf7c2429659cbab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16431
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Supporting SRQ caused two kinds of memory leaks. Fix both in this patch.
1. rqpair->rsps was leaked and null pointer access occurred
An error was detected during the nightly nvmf_delete_subsystem test.
The NVMe perf tool crashed with SIGABRT.
The reason of the crash was
nvme_rdma.c:2504:2: runtime error: member access within null pointer of type 'struct nvme_rdma_rsps'
This was caused by clearing rqpair->rsps before freeing rqpair->rsps.
rqpair->rsps should have been held until rqpair->rsps is freed. However,
when we support SRQ, rqpair->rsps was cleared when releasing rqpair->poller
by mistake. rqpair->rsps should be cleared only if SRQ is enabled because
in this case rqpair uses rsps of rqpair->poller.
2. rqpair->reqs and rsps are leaked for admin qpair at controller reset
To avoid unnecessary alloc and free for rqpair->rsps when enabling SRQ,
nvme_rdma_create_reqs() and nvme_rdma_create_rsps() were moved to
nvme_rdma_connect_established().
On the other hand, nvme_rdma_free_reqs() and nvme_rdma_free_rsps() were
called by nvme_rdma_ctrlr_delete_io_qpair().
However, at controller reset, admin qpair was just disconnected and
reconnected. In this case, nvme_rdma_create_reqs() and
nvme_rdma_create_rsps() were called again without calling
nvme_rdma_free_reqs() and nvme_rdma_free_rsps().
Hence, memory leak occurred.
To fix the memory leak, move nvme_rdma_free_reqs() and nvme_rdma_free_rsps()
from nvme_rdma_ctrlr_delete_io_qpair() to nvme_rdma_qpair_destroy().
One of the fixes fot the issue #2874
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I167ba908cff73d7a0be2248affce4c54f233da51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16384
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This routine was neglecting to reset ->iovcnt, leading to havoc when the
request was re-used.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ifd4ac47b95edd517ce5df731c682697bf51da819
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16273
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For IC_RESP and C2H_TERM_REQ, use the regular async write path but add
an additional flush. The flush operation reports errors, so we may at
some point attempt to handle busy conditions by blocking. However, for
now this doesn't do that because previously the writev call didn't
either.
Change-Id: I4d05e19ac6dd781be7c96005549abdb52511b8c1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15213
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
If we call flush, we want to flush regardless of whether there is a poll
group.
Change-Id: I88680105d999a909f3f1fe75be9caff31a8555ff
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16420
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
When a queue has finished processing on its polling
thread, it sends a message to the app thread signaling
that it is done. Then when the app thread gets
messages from all of the queues for that device, it can
proceed with tearing the device down.
But if there are still ctrl_ring commands in progress,
it needs to wait. Previously it would register a
poller that would retry the same function if it
found commands in progress. But the problem is that
it did not differentiate the function getting called
as a direct message from the polling thread vs. retried
via the poller on the app thread. This could result
in lost messages.
So fix it to always increment the queues_closed
counter (renamed from q_deinit_num), and then
only check for ctrl ring commands in progress after
we received all of the queue closed messages.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0ea23ebc69acb29d5ab7e1d86ddbe74b9973e225
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16405
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
We make a few changes here to enable this:
1) Set IORING_SETUP_SQPOLL on the control ring.
Otherwise when UBLK_START_DEV is submitted it
will be processed in the context of the system
call itself, resulting the kernel block layer
submitting reads to the new device which blocks
the thread - meaning the system call may never
return.
2) Save the cmd_op in each sqe, along with the
spdk_ublk_dev pointer.
3) Add a poller to poll the ctrl ring. The poller
can get the ublk and cmd_op from the cqe to
know which ctrl operation completed and take
next steps as necessary.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia0e51a4ff74781c85967c54969fbfc67a0d3f115
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16404
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There is a way to pause/resume spdk pollers, however there is no way
to achieve that using public API for the given target which has
a hook behaving similar to pollers. Exposing such functionality can
be used for pausing and restoring target pollers during
reset, e.g. new commands should not be fetched to assure
that all internal resources can be cleared/reinitialized safety.
Pausing target poller during the reset will assure that, without
need for destroying transport or adding condition statements in IO path.
Similar use case might be hitless upgrade. Depending on implementation
there might be need that no new command can be submitted when
secondary processes are being switched to upgraded versions.
Pausing target pollers should be useful in this case.
Signed-off-by: Kamuda Szymon <szymon.kamuda@intel.com>
Change-Id: I419816552c710c43e02197ebcc20a967fb23b3bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.
Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.
Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.
This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests
Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +
Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
With per channel stats called on thread A,
spdk_bdev_for_each_channel calls
spdk_for_each_channel which immediately sends
a message to thread B.
If thread B has no workload, it may execute the
message relatively fast trying to write stats to
json_write_ctx.
As result, we may have 2 scenarious:
1. json_write_ctx is still not initialized on
thread A, so thread B dereferences a NULL pointer.
1. json_write_ctx is initialized but thread A writes
response header while thread B writes stats - it leads
to corrupted json response.
To fix this race condition, initialize json_write_ctx
before iterating bdevs/channels
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Reported-by: Or Gerlitz <ogerlitz@nvidia.com>
Change-Id: I5dae37f1f527437528fc8a8e9c6066f69687dec9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16366
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a preparation for the next patch to fix the race condition of
per channel mode.
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I9eaefc527ccf82011af39b8261f5b3cc12983bda
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16365
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For now, we will still execute the two parts
consecutively and synchronously. Follow-up patches
will do the second part asynchronously, after the
ublk cmds associated with the first part have
completed.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I814d885a8a113c3367207d11ae09dd536eb63460
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16403
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Upcoming patches will submit ctrl cmds and wait for
them to complete asynchronously. So we will want to
first send the ADD_DEV and SET_PARAMS commands, wait
for them to complete, and only then open the ublk
device file.
So to prepare for that sequencing, move the open()
from _ublk_start_disk to ublk_start_disk.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9fdf19ce9b51bd552faa917e1e842f9ddfb111a1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16402
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We will put this pointer into the sqe. It will
be useful when we start doing async completions
on the ctrl ring.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3bdb728eb1d3ed66a8ecd05df208e4f36e3fbe0b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16401
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When blobstore is shutdown unexpectedly the super block should
already be marked as dirty. Only when proper blobstore unload
happens, the super block is marked as clean.
Super block is not marked as dirty on blobstore load,
but on first action that starts to modify the metadata.
At this time it only happens through blob persist,
which is fine for creation/deletion of blobs or
their modification (resize/xattr).
It works for cluster allocation with extent_table disabled,
and when extent page needs to be allocated.
Yet it fails for cases when no new extent page is required.
It will result in not marking blobstore as dirty and
then fail when loading a particular blob due to mismatch
between used_clusters and contents of extent page.
To fix that, the blobstore is now marked dirty on a very first
extent page update since blobstore load.
Fixes#2830
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ied37ecf90d46e1bc51b22c323dce278a0fa88f72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Currently we do not have a way to dump opts
for virtio_blk transports. This patch introduces
necessary changes to let us save and load those
via JOSN config.
Change-Id: I7ee4f31062f3d4a264f322e66a67ba3d075f1d75
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5b8ce0a4571872e6755c5fa0abbfa1a981dd411f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16400
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch makes marking blobstore as dirty, separate from
persist process. Adding a context strucutre and new single
callback once completed.
It is being as part of refactor to allow marking blobstore
dirty when writing out extent page - see #2830.
Change-Id: Ie2e9cc32860697e0e747939842ab04f48fbff49b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Patch https://review.spdk.io/gerrit/c/spdk/spdk/+/16328,
introduced a refactor for persist path. No functional change
should occur with it, but the code layout after compilation
(path length) might have changed.
It resulted in unrelated scan-build failure:
https://ci.spdk.io/results/autotest-per-patch/builds/95746/archive/scanbuild-vg-autotest/scan-build/report-d08e76.html#EndPath
Tried to replicate the issue without the above patch,
by increasing maxloop or -analyze-headers in scan-build.
Didn't result in any new failures in blobstore.
This seemed like a false positive, so it was verified:
https://review.spdk.io/gerrit/c/spdk/spdk/+/16333/
With no other options, an assert is added only to the
function where the false positive occured.
scan-build log for posterity:
blobstore.c:1062:58: warning: Division by zero [core.DivideZero]
desc_extent_rle->extents[extent_idx].cluster_idx = lba /
lba_per_cluster;
~~~~^~~~~~~~~~~~~~~~~
blobstore.c:1079:58: warning: Division by zero [core.DivideZero]
desc_extent_rle->extents[extent_idx].cluster_idx = lba /
lba_per_cluster;
~~~~^~~~~~~~~~~~~~~~~
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9dd729fa13ce1c9bbcb91e4326658e2b4e326e6d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16335
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The iobuf buffers are only used in accel when executing chained
operations. Since none of the components in SPDK are using chaining
yet, there's little point in having per-thread iobuf caches, as they
only reduce the number of available buffers in other libraries.
This change will be reverted once bdev layer and bdev modules are
updated to support chaining.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ibad19ea92f2218a8dec01e802a736cfdd357dfc6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16398
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This allows to get start address and length of each
memory chunk in order to create app-specific
resources.
Since we don't want to expose rte structure in the
callback, we have to remap rte data types to SPDK.
Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I3865c4cfe532c6a99a5a3c6c983ded8b9a338de1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
directly
This patch removes hardcoded compressdev code from the
vbdev module and instead uses the accel_fw. The port required
a few changes based on how things are plumbed and accessed,
nothing that isn't be too obscure. CI tests were updated to
run ISAL accel_fw module as well as DPDK compressdev with QAT.
Unit tests for the new module will follow in a separate patch.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I769cbc888658fb846d89f6f0bfeeb1a2a820767e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13610
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
ublk could export a backend device as ublk block device (/dev/ublkb*).
A rpc method is used to add ublk device and it should be done
after creating ublk target. Corresponding, ublk_del_dev is
used to delete the specified ublk device.
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Co-authored-by: Xiaodong Liu <xiaodong.liu@intel.com>
Change-Id: I3a4ba8d8dc5f5ad241511ccbc9d3336b582a6dc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
