Commit Graph

2119 Commits

Author SHA1 Message Date
Jin Yu
30f9feca2a test: add md_buf unit test in splitting test
Change-Id: I83010cdc7af49a29e516bcd351d67154307756ad
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-11-11 00:55:08 +00:00
Madhu Adav MJ
fce9428772 nvmf: Async event support for discovery log change
Added asynchronous event notices for discovery log change
as per nvme fabrics spec 1.1. This allows a host with persistent
connection to discovery controller to automatically connect to any
new subsystem available to the host automatically.

According to nvme fabrics spec 1.1, if the connect command specifies a
non-zero keep alive timer value and the discovery controller does not
support asynchronous events then we need to return Connect Invalid.
Since SPDK does not implement this check instead added support
for asynchronous events in discovery controller.

Change-Id: I4cade5f7d24826ce97a2fa2b4ca688a1d728c1db
Signed-off-by: Madhu Adav MJ <madhu.adav@flipkart.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
2020-11-11 00:53:57 +00:00
Jim Harris
0e47be4b41 nvme: add nqn prefix to hostnqn
Section 7.9 of the NVMe spec says that all nqns must
start with "nqn.".

Fixes issue #1669.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7fd0e6a0a397e831c4fa2377126b6b1e1b127d88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5017
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-11-06 08:11:54 +00:00
Jim Harris
e7cbcec38e app_ut: add stubs required w/ clang build
clang-9 is a bit more onerous on stubs required in
the unit test file.  app_ut specifically needs
spdk_reactors_init and spdk_reactors_fini added.

Fixes issue #1613.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa219829e933405ddb662f5b991204ef05b3bd75
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-11-05 09:38:48 +00:00
Jim Harris
42ec403b6a test: add some missing stubs
This fixes link issues with clang on some systems.

Fixes another part of issue #1613.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icd51351b8e38043f267918b85e8ed1c3daab2db7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-11-02 11:23:21 +00:00
Liu Xiaodong
5568355eb0 vhost-blk: apply interrupt
If interrupt mode is set, related poller functions will be
registered to interrupt handler instead of poller.
interrupt_tgt can work as vhost-blk target to support VM.

Change-Id: I3a15f9a63532f44fe0d2f0cb69b0efdd72431d10
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-11-02 11:22:40 +00:00
Michael Haeuptle
7fc48a5ffc nvme: break completion loop when ctrlr is invalid
This fixes #1423 where the completion loop never
breaks when the NVMe ctrlr is no longer present.
This condition can happen during a hot remove.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: Ia238c8aeae720832068de28ce4d34a9d233344fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4831
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 08:11:35 +00:00
Jim Harris
ddf86600bb nvme: continue probing ctrlrs even if one fails
It is possible that a single probe_ctx could be used
to probe multiple newly attached nvme controllers.  If
one of those controllers is removed during this process,
the rest of the controllers do not get probed and can
even get stuck in a zombie state.

It is better to just continue with probing the rest of
the controllers.

Fixes issue #1611.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4156ee8b50e8d52cfeee7224f210a58bb773e939

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4945
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-29 08:11:17 +00:00
Alexey Marchuk
85fa43241b nvmf/tcp: Support ICD for fabric/admin commands
According to the SPEC we should support up to 8192 bytes
of ICD for admin and fabric commands. Transport configuration
parameter in_capsule_data_size is applied to all qpair types -
admin and IO. Also we allocate resources when we get a connection
request, so we don't know qpair type at this moment.
Create a list of buffer in TCP poll group to support ICD up
to 8192 bytes when configuration ICD is less than this value.
The number of elements in this pool is hardcoded, it is planned
to add a new configuration parameter later.

Fixes issue #1569
Change-Id: I8589e3e2ea95d515f5503c6de7c1ee40aaf7b6da
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4754
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-27 08:40:12 +00:00
Alexey Marchuk
e385cafa72 nvme: Don't log an error when we can't resubmit all requests
In TCP NVME initiator with zero copy enabled requests might be
completed asynchronously - out of qpair_process_completions
context. At the same time we calculate requests completed
asynchronously so that generic NVME layer can resubmit
queued requests after calling qpair_process_requests (or
poll_group_process_requests).
But there is a time gap between async request complete and
qpair_process_completions and the user can submit new IO
thereby decrease the number of free TCP requests. That means
that there might be less free requests than we excpected when
we try to resubmit queued requests.
The solution is change ERRLOG to DEBUG log since it is not a
fatal case.

Change-Id: If045ecd331cc6693e8ef450d8e15432dfa5d8812
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4859
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-10-26 11:48:46 +00:00
Vitaliy Mysak
7148f33360 event: adopt static scheduler to the new API
Move current scheduler to it's own module
and make use of new API.

Change-Id: I4928aed82603d51de01194c9650709e814f7f61b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4054
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Tomasz Zawadzki
02d4c2b32a ut/iscsi: remove unused legacy config files
The tests did not use the legacy files anyway.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Icffedb7cc7a8574f0f801883b2d9655515cab687
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:12:18 +00:00
paul luse
33eac886b9 lib/idxd: refactor batching for increased performance
And to eliminate an artificial constraint on # of user descriptors.
The main idea here was to move from a single ring that covered all
user descriptors to a pre-allocated ring per pre-allocated batch.

In addition, the other major change here is in how we poll for
completions.  We used to poll the batch rings then the main ring.
Now when commands are prepared their completion address is added to
a per channel list and the poller simply runs through that list
not caring which ring the completion address belongs too. This
simplifies the completion logic considerably and will avoid
polling locations that can't potentially have a completion.

Some minor rework was included as well, mainly getting rid of the
ring_ctrl struct as it didn't serve much of a purpose anyway and
with how things are setup now its easier to read with all the
elements in the channel struct.

Also, a change that came in while this was WIP needed a few fixes
to function correctly.  Addressed those and moved them to a
helper function so we have one point of control for xlations.

Added support for NOP in cases where a batch is submitted with
only 1 descriptor.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie201b28118823100e908e0d1b08e7c10bb8fa9e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-22 22:43:28 +00:00
Tomasz Zawadzki
78ad672861 lib/event: remove legacy configuration from event framework
All options -c, --config and --json are valid, but have to
point to JSON configuration file.

Adjusted UT since JSON configs don't work with --wait-for-rpc.

Since this removes last reference to legacy INI configuration,
updated conf library to no longer mention the deprecation.
All uses of conf library are for explicit reason and not
related to SPDK event framework configuration.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic9a702465982daf715ce1c2ab863c48584734611
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-22 17:08:41 +00:00
Monica Kenguva
74f8f3712c unit: add a UT for the get_ana_log_page
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Ic1edb61697ba1e9a22bfde53e5044ec6bb6a80d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3903
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
2020-10-22 03:56:35 +00:00
Tomasz Zawadzki
6511969c3b event: remove subsystem legacy config print
This patch removes callback function for subsystems to
present their options for legacy config.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I63076fc03eff45da5d57fab03501602922a20e3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4749
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-21 02:25:45 +00:00
Ivan Betsis
c818233b41 nvmf/rdma: Add WR batch rdma new transport option
With x86 and low queue depths with multiple QPs/initiators
there is a benefit from disable batch when we have randread IO pattern.

Testing environment: x86, Intel(R) Xeon(R) Silver 4116 CPU @ 2.10GH
Several results:
init_cores | QD | BS | Avg IOPs/BW without batch | Avg IOPS/BW with batch
  8 cores  | 4  | 4K |     1870087 / 7305        |     1594014 / 6226
  8 cores  | 8  | 4K |     1853573 / 7240        |     1576400 / 6157
  8 cores  | 16 | 4K |     1819643 / 7108        |     1569487 / 6130
  8 cores  | 32 | 4K |     1815467 / 7092        |     1569909 / 6132
 16 cores  | 4  | 4K |     1908018 / 7453        |     1566843 / 6120
 16 cores  | 8  | 4K |     1906081 / 7446        |     1562110 / 6102
 16 cores  | 16 | 4K |     1880706 / 7346        |     1555060 / 6074
 16 cores  | 32 | 4K |     1835878 / 7171        |     1548156 / 6046

Signed-off-by: Ivan Betsis <c_ivanb@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Change-Id: Icdbbbdf83f137eb4f05bd2063268ee2a7d87335a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/924
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-21 02:25:32 +00:00
Shuhei Matsumoto
ea1bfd84cd lib/nvme: Make internal of spdk_nvme_detach() asynchronous
Add two new helper functions, nvme_ctrlr_detach_async() and
nvme_ctrlr_detach_poll_async() to make the internal of
spdk_nvme_detach() asynchronous.

Use callback function to remove controller from the attached list after
completing shutdown and before freeing to avoid conflict between
attach and detach.

Update MOCKs in the corresponding unit test cases.

The next patch will add two public APIs spdk_nvme_detach_async()
and spdk_nvme_detach_poll_async() based on this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifbdfec2a1facde9354007c6248f280e245a36eed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 16:55:55 +00:00
Tomasz Zawadzki
6d86242dc5 subsystem/iscsi: remove legacy config support
This patch removes legacy config support in iscsi subsystem.
All options through the legacy config are already reflected in JSON.

Following RPCs are corresponding to removed legacy config sections:
Initiator groups - iscsi_create_initiator_group and iscsi_initiator_group_*
Subsystem options - iscsi_set_options
Portal groups - iscsi_create_portal_group
Target node - iscsi_create_target_node and iscsi_target_node_*

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I11326a84d4d580b19db422b8522198eea5a5be0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 08:53:53 +00:00
Shuhei Matsumoto
2842056073 blobfs_bdev: Use spdk_bdev_create_bs_dev_ext() instead of spdk_bdev_create_bs_dev_from_desc()
Replace all calls of spdk_bdev_create_bs_dev_from_desc() and
spdk_bdev_open_ext() by calls of spdk_bdev_create_bs_dev_ext()
including unit tests.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idb9dc5a10f6e221c26e82e0194930cb7a2071dae

Change-Id: I61e577db9e26ef3e1c3e2e4093ad66922a178f34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4723
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-20 08:52:29 +00:00
Shuhei Matsumoto
b6cbf872eb bdev/lvol: Pass bdev name instead of bdev pointer to vbdev_lvs_create
Change the first parameter of vbdev_lvs_create() from bdev pointer
to bdev name, and then remove spdk_bdev_get_by_name() from
rpc_bdev_lvol_create_lvolstore().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9b0e47f551657f6aa1235af37a8357b0af62d414
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4707
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Shuhei Matsumoto
6051368da9 bdev/lvol: Use spdk_bdev_create_bs_dev_ext() when creating lvol store
This is a drop-in replacement. The following patches will remove
spdk_bdev_get_by_name() from the operation to create lvol store.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I31688cf98625f1d13012281d89f06d343e2f2cef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Shuhei Matsumoto
94d6e332bc ut/nvmf_ctrlr_discovery: Remove spdk_bdev_open() stub because it is not used
Remove the stub for spdk_bdev_open() and use the DEFINE_STUB_V macro
to the stub for spdk_bdev_close().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I84df06b4b73e1f5af5c2c8854f1613e8a413533b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4729
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Shuhei Matsumoto
6a684a349e ut/bdev_mt: Use spdk_bdev_open_ext() instead of spdk_bdev_open()
Replace all spdk_bdev_open() calls by spdk_bdev_open_ext() calls
in the multiple SPDK threads cases of the unit tests for lib/bdev.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I65591f12f8737bb8a0c1256fa84c5b6dac481150
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4728
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Shuhei Matsumoto
75dfecbb16 ut/bdev: Use spdk_bdev_open_ext() instead of spdk_bdev_open()
Replace most spdk_bdev_open() calls by spdk_bdev_open_ext() calls
in the unit test for lib/bdev.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifbb8a7338b1d99808c96c87be1cfbee145b0d21f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4727
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Tomasz Zawadzki
5b1e154a5c bdev: remove legacy config support
This patch removes legacy config support in bdev layer.
All options through the legacy config are already reflected in JSON.

Removed configuration can be set via `bdev_set_options` and
`bdev_set_qos_limit` RPC.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I24c365625540659cad425268d2aa41e3bf279d5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4645
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
ceea308887 lib/blobfs: remove legacy config support
This patch removes legacy config support in blobfs library.
Mostly reverting patch:
(97f3104) blobfs: Add conf parse for blobfs.

CacheBufferShift option in legacy config is not tested in CI,
and never received corresponding RPC.
If required the RPC can always be added later on.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I54e39f069047a243b2186ae5ea225ed452180488
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4668
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
269efef886 bdev modules: remove legacy config support
This patch removes legacy config support in:
 crypto bdev module
- nvme bdev module
- ocf bdev module
- rbd bdev module
- pmem bdev module
- iscsi bdev module
- raid bdev module
All options through the legacy config are already reflected in JSON.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I00213365f52d3de1012493c14d4ea5fc537e595e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4673
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-20 02:42:16 +00:00
Changpeng Liu
6bd7bd0da2 vhost: deprecate internal vhost library support
The internal vhost library is used when DPDK's version
is older than 19.05 and the experiemntal vhost nvme
target.

For the CONFIG_INTERNAL_VHOST_LIB option, SPDK doesn't
enable this option by default over one year and CI
doesn't cover it either, so we may remove it with
this release.

As for the vhost-nvme target, since we are developing
a new vfio-user target solution, it's OK for us to remove
it now.

Change-Id: Ib2cce1db99cd09754307c2828b3187f2d4550304
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-20 02:42:16 +00:00
Darek Stojaczyk
c77c655987 json: add spdk_json_free_object()
After decoding a JSON object we had to free the parsed
strings one-by-one. Not anymore.

Change-Id: I819f1d533e397aa9babca58b5500c38ac01a963d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2753
Reviewed-by: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-19 10:02:10 +00:00
Jim Harris
44ff994386 event: remove include of spdk_internal/thread.h
It's not needed.  Add it to one of the UT where it
is needed though.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I06628184c22df68deffce3ca0561878569ccf3f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4717
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-19 10:00:29 +00:00
Jim Harris
fcaf67e238 test/rdma: fix array subscript violation in rdma_ut.c
Found by gcc 10.1.1.

Fixes issue #1623.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b43b0ad1db5048cadaccc16e3c9a47b2592aa6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4715
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-19 10:00:29 +00:00
Shuhei Matsumoto
229b996850 bdev/ocssd: Removed unused ioch parameter from bdev_ocssd_get_zone_info()
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4a7a79892ca8d6010888e69d095e58c4b50bafce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4526
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Mellanox Build Bot
2020-10-19 10:00:16 +00:00
Darek Stojaczyk
f766d1e478 nvmf: let transports parse transport opts from JSON themselves
There are more transport on the way and we don't want to add
all their various opts into the single, generic structure.
We'll pass the JSON structure to transports instead. Then
the transport code can custom pull from the JSON any param
it wants.

To complement that, transports will now also have their own
JSON config dump callback. This was only done in the generic
nvmf.c so far, with conditions for RDMA and TCP.

Change-Id: I33115a8d56cec829b1c51311a318e0333cc64920
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-19 09:55:05 +00:00
Shuhei Matsumoto
7a85820db3 bdev/zone: Use spdk_bdev_open_ext() instead of spdk_bdev_open()
This is just a drop-in replacement.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I018ef60648122aa672430bb62dc7b1a2e1cd5d7c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4648
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-16 23:12:03 +00:00
Shuhei Matsumoto
d8a1892422 bdev/raid: Use spdk_bdev_open_ext() instead of spdk_bdev_open()
This is a drop-in replacement. Update unit test together, and the
idea of the update was from the unit test for zone bdev module.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia8d0644f42b9d0a0ad502eebbe3e414abd1de4cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4591
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-16 23:12:03 +00:00
Shuhei Matsumoto
491e6c4309 bdev/crypto: Open base bdev first using spdk_bdev_open_ext()
vbdev_crypto_claim() gets bdev name instead of bdev pointer
as a parameter, and open the corresponding base bdev first using
spdk_bdev_open_ext().

The purpose is to fix the race condition due to the time gap
between spdk_bdev_get_by_name() and spdk_bdev_open(). A bdev pointer
is valid only while the bdev is opened.

Resize event is not supported for now.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia0e1ce2ce696f431bb26af94729931c3ffb9a9d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4588
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-16 23:12:03 +00:00
yidong0635
9714818d04 test/bdev_ut: fix warning about unitialize value.
gcc version 10.1.1 20200619 releases/gcc-10.1.0-223-gf8e16e23d0
(Clear Linux OS for Intel Architecture)

Issue reports:

  CC test/unit/lib/nvme/nvme_ns_cmd.c/nvme_ns_cmd_ut.o
bdev_ut.c: In function ‘bdev_io_alignment_with_boundary’:
bdev_ut.c:2146:2: error: ‘buf’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
 2146 |  free(buf);
      |  ^~~~~~~~~
bdev_ut.c: In function ‘bdev_io_alignment’:
bdev_ut.c:2008:2: error: ‘buf’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
 2008 |  free(buf);
      |  ^~~~~~~~~

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I6d1bbf74b230b7bc51646454ddba0db4792f5900
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4693
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-16 08:15:57 +00:00
Tomasz Zawadzki
7f139e549b lib/blob: extent_page shall never occupy md page 0
Search for md_page to be used as extent page started
from 0, which is completely valid md_page.

This page can be free when for example blob with id 0
was deleted and some other requested a new page for extent.

There are already existing blobs that have extents
pointing to 0, which means unallocated.
Unfortunetly it means 0 can never mean md page 0.
If that already occured for someone, this extent page was already
lost during blob/bs reload and nothing can be done.

With this in mind following assumptions are made for extent pages:
- 0 means unallocated extent page
- UINT32_MAX means we ran out of md pages, and should not be persisted
- [NEW] extent page can never occupy md page 0

That last one is new addition in this patch.
bs_allocate_cluster will now always try to find md page from 1 or higher.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia17ce5bbca2fab4fb4487e4e263f3a0aa120bf17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4314
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-16 08:15:16 +00:00
Tomasz Zawadzki
63e257ed34 lib/blob: update pages array during blob load
When loading a blob and parsing its metadata,
the array of pages was not updated.
Serialization was unaffected, since the current pages
array is unused there.
Behavior was working correctly for first page,
but did not for any blob with more than one page.

Unfortunetly blob_persist_zero_pages() never zeroed out
the pages, neither blob_persist_zero_pages_cpl()
released the md pages.

Resulting in md pages being claimed even after blobs
deletion.

This patch now fills out the active pages array with
the appropriate page numbers from metadata.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6ff1f4fe95684119d283c2471fdbbea464da8151
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-16 08:15:16 +00:00
Shuhei Matsumoto
aec30063e3 bdev/compress: Pass bdev name instead of bdev pointer to _prepare_for_load_init()
This patch removes the call of spdk_bdev_get_by_name() from
create_compress_bdev() by calling spdk_bdev_open_ext() and then passing
bdev descriptor to _prepare_for_load_init().
vbdev_init_reduce() returns the return code of spdk_bdev_open_ext()
directly if it is error to process the ENODEV case correctly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie80ce4104ceaa5726db976e33cd35352d5c91703
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4571
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-16 08:14:34 +00:00
Tomasz Zawadzki
4e8e97c886 log: remove internal log.h header
There is nothing left here, so remove it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib947d42bc577dbebb4650b1be885e05a80f8f8cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
2020-10-15 08:23:39 +00:00
Shuhei Matsumoto
779a6bdf82 bdev/compress: Replace spdk_bdev_open() by spdk_bdev_open_ext()
This patch is just a drop-in replacement.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iffb253ba3a46f03f9927c4ba4206376509cbb5fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4566
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2020-10-14 08:07:06 +00:00
Shuhei Matsumoto
a22d8a530f lib/nvmf: Add an API spdk_nvmf_subsystem_add_ns_ext() to pass bdev_name
Add an new API spdk_nvmf_subsystem_add_ns_ext() to pass not bdev but
bdev_name to fix the race condition due to the time gap between
spdk_bdev_get_by_name() and spdk_bdev_open(). A pointer to a bdev is
valid only while the bdev is opened.

spdk_bdev_open() has been replaced by spdk_bdev_open_ext() but the
issue still existed.

Update the corresponding unit tests accordingly.

Then replace the internal of spdk_nvmf_subsystem_add_ns() by
spdk_nvmf_subsystem_add_ns_ext() call.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifcaa2121129ef22d5e61c9a8f7c640ff37a64485
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4485
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-14 08:07:06 +00:00
Tomasz Zawadzki
2172c432cf log: simplify SPDK_LOG_REGISTER_COMPONENT
This patch removes the string from register component.
Removed are all instances in libs or hardcoded in apps.

Starting with this patch literal passed to register,
serves as name for the flag.

All instances of SPDK_LOG_* were replaced with just *
in lowercase.
No actual name change for flags occur in this patch.

Affected are SPDK_LOG_REGISTER_COMPONENT() and
SPDK_*LOG() macros.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I002b232fde57ecf9c6777726b181fc0341f1bb17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
2020-10-14 08:00:35 +00:00
Tomasz Zawadzki
849aa41d71 log/ut: use register macro in UT
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib42a3aa4f44ef7a5e55f0d59441f24736bebf735
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Mellanox Build Bot
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-14 08:00:35 +00:00
Tomasz Zawadzki
617a55953c log: fix register macro ending with semicolon
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I98bf7620da9065e9ab0ac3ba0444ba5998e69b8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mellanox Build Bot
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
2020-10-14 08:00:35 +00:00
Ben Walker
d48a7b26b3 env: spdk_vtophys buffer argument is now const
Indicate that this parameter is not modified.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I1e68f6c5ed4cbfaed0495e969ee1a50c31e9ccac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-13 07:48:05 +00:00
Ben Walker
5e4e4bc485 nvmf: Group bool members of spdk_nvmf_subsystem into a bit field
This saves 2 bytes and allows it to pack nicely with the
changing state bool (which must remember separate for atomic
operations).

Change-Id: Ibb92ae3c74306e60385ae23d0aaf877f33a69095
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4553
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-09 07:18:49 +00:00
Shuhei Matsumoto
ab808d2b68 lib/scsi: Pass bdev_name instead of bdev to scsi_lun_construct()
Pass not bdev but bdev_name to scsi_lun_construct() to fix the
race condition due to the time gap between spdk_bdev_get_by_name()
and spdk_bdev_open(). A pointer to a bdev is valid only while the
bdev is opened.

spdk_bdev_open() has been replaced recently by spdk_bdev_open_ext(),
but the issue still existed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic462422dbc2501c24907f56a36570fbb54fef65b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4482
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-08 13:11:15 +00:00
Anil Veerabhadrappa
2fc3ef3231 nvmf/fc/ut: Call spdk_nvmf_tgt_destroy() in the correct thread context
Use the same thread context to call both spdk_nvmf_tgt_create() and
spdk_nvmf_tgt_destroy(). This is required to ensure the accept poller
is registered and unregistered in the same thread context.

Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I0637c7896ca7504412fbe673355f6904dd81a961
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-30 07:26:13 +00:00
Niklas Cassel
c4d1b7d57a nvme: add new state to get Identify IOCS Specific Namespace data structs
Add a new state in the SPDK NVMe state machine in order to fetch
I/O Command Set Specific Namespace data structures.

Right now there is only support for the Zoned Namespace Command Set
Specific Identify Namespace data structure.

The NVM Command Set Specific Identify Namespace data structure is
all zeroes right now, reserved for future use.
The Key Value Command Set Identify Namespace data structure is not
all zeroes, however, adding support for Key Value is outside the
scope of this patch.

The new NVME_CTRLR_STATE_IDENTIFY_NS_IOCS_SPECIFIC state is added
after the NVME_CTRLR_STATE_IDENTIFY_ID_DESCS state. This is because
we need to have fetched the identifiers in the desc list in order
to know which command set a namespace belongs to.

A slightly nicer design might have been to refactor the NVMe state
machine to first fetch the id desc list, then the identify namespace
struct, and finally the identify IOCS specific namespace struct.
However, since this would have required a lot of changes, it didn't
really seem justified.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I62cbc533c2c3eec1ccf0ba9b1c414d5a70919cff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4368
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-09-30 07:26:04 +00:00
Niklas Cassel
64563ada5d nvme: add new state to get Identify IOCS Specific Controller data structs
Add a new state in the SPDK NVMe state machine in order to fetch
I/O Command Set Specific Controller data structures.

Right now there is only support for the Zoned Namespace Command Set
Specific Identify Controller data structure.

The NVM Command Set Specific Identify Controller data structure is
all zeroes right now, reserved for future use.
The Key Value Command Set Identify Controller data structure is also
all zeroes right now, reserved for future use.

The new NVME_CTRLR_STATE_IDENTIFY_IOCS_SPECIFIC state is added
after the NVME_CTRLR_STATE_IDENTIFY state. That way, if support for
the Zoned Namespace Command Set is enabled during probing, we will
fetch the Zoned Namespace Command Set Specific Identify Controller data
structure, regardless if any Zoned Namespaces are attached or not, and
no additional steps will be needed once a Zoned Namespace is attached.

Since we only have one command set to fetch, avoid creating
NVME_CTRLR_STATE_IDENTIFY_IOCS_SPECIFIC substates, although that will
probably be needed when support for another command set is added.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I95535b09b03b7ef2ee9a11eebdbd28aad66d65ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4367
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-30 07:26:04 +00:00
Alexey Marchuk
bc36528cda nvme/tcp: Process poll_group when waiting for icresp
A preparation step for enabling zero copy in NVMEoF TCP initiator.
Since nvme_tcp_qpair_process_completions doesn't process poll
group, we can't get asycn notification from kernel.

1. Add a qpair to poll group before we send icreq in order to be able
to process buffer reclaim notification.

2. Check if qpair is connected to a poll group and call
nvme_tcp_poll_group_process_completions instead of
nvme_tcp_qpair_process_completions when waiting for icresp

3. Add processing of poll group to nvme_wait_for_completion_timeout
and nvme_wait_for_completion_robust_lock since they are used to
process FABRIC_CONNECT command

Change-Id: I38d2d9496bca8d0cd72e44883df2df802e31a87d
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-09-29 09:35:47 +00:00
Li Feng
5029fe14b6 scsi: add bdev resize callback support
Currently, the scsi bdev only supports the hotremove event,
and the scsi library uses the deprecated `spdk_bdev_open` function.

In this patch, add the resize event support, so the upper layer
could do more actions, like vhost-scsi could notify the guest os.

For the scsi compatibility, add _ext suffix for some public api.

Change-Id: I3254d4570142893f953f7f42da31efb5a3685033
Signed-off-by: Li Feng <fengli@smartx.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-09-28 08:33:18 +00:00
Shuhei Matsumoto
06d25b7021 lib/nvme: Add an internal API nvme_wait_for_completion_robust_lock_timeout
Add an internal API nvme_wait_for_completion_robust_mutex_lock_timeout()
and related internal APIs just call it with adjusting parameters.

nvme_wait_for_completion_robust_lock_timeout() will be usable for
the current use cases of nvme_wait_for_completion_robust_lock() and
future use cases.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2b499643930256a39ebe279f56a399f20a7a2fde
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4217
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-09-25 11:42:44 +00:00
Jim Harris
47c45d243a test: do not try to execute blob_ut if it doesn't exist
We do not compile blob_ut on systems with Cunit version
that is too old.  So modify unittest.sh for this case
specifically, so that users with an older Cunit version
can still run the rest of the unit tests.

Fixes issue #1601.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifd226a47a2ff712ad1f14fd12e1cb796e44be6b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4372
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-25 11:29:55 +00:00
Niklas Cassel
d1db020954 nvme: add a csi parameter to nvme_ctrlr_cmd_identify()
With the introduction of namespace types, the identify command has
gained an additional parameter: Command Set Identifier (CSI).

This parameter is similar to the existing parameters NSID and CNTID,
and is not used by all CNS values.

Most notably, the CSI parameter is not used for the existing CNS
values 00h (ID NS) and 01h (ID CTRL).
There are new CNS values, e.g. 05h (ID IOCS specific NS), and
06h (ID IOCS specific CTRL), which do take the new CSI parameter.

The new CNS values instead return Command Set Specific data structures,
which is basically an additional data structure. Therefore, the CNS
values 00h and 01h are very much still in use.
(Even the NVM Command Set has a Command Set Specific data structure,
even though all fields in that data structure are currently reserved.)

Since the CSI parameter is unused by all the existing calls to
nvme_ctrlr_cmd_identify() (since none of the calls send in a CNS value
that uses CSI), simply send in 0 for all existing calls.
No functional change intended.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ia2b2324393a0707152b2f8511f0a22ad4a12bd46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4309
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-09-22 11:39:53 +00:00
Niklas Cassel
726ac67c55 nvme: use the most appropriate is_active() function
There are two different is_active() functions.
spdk_nvme_ctrlr_is_active_ns() which iterates through the active_ns_list,
and spdk_nvme_ns_is_active(), which simply checks the nsdata.
There is an event callback that refreshes active_ns_list when a relevant
events has occured.

In nvme_ns_construct(), nvme_ctrlr_identify_ns() has just been called,
so we know that nsdata is as fresh as possible.

Hence, there is no reason to iterate through a less fresh active_ns_list.
Since we know that the nvme_ctrlr_identify_ns() call was done through the
same controller, we also know that the active/inactive is from the
perspective of the correct controller, so that is not a reason to use the
less efficient is_active() function.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I185f59b53e16e70163e33a3909f4b55ebf631cc4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4293
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-18 07:25:06 +00:00
Niklas Cassel
743271d92d nvme: cache the command set identifier
Since the command set identifier might be accessed at several
different states in the nvme state machine, cache it so that
we don't need to loop through the ns id desc list every time.

This is similar to how other identify fields are cached using
nvme_ns_set_identify_data().

None of the identifiers in the desc list (including the new CSI)
can change over the life time of a namespace, so caching them
should be safe.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ie06180a4b3750dfa1a42f47afe0f7f9e3ec04ba9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4266
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-18 07:25:06 +00:00
Niklas Cassel
c31fc45324 ut/nvme: use nvme_ns_construct()/nvme_ns_destruct() in nvme_ns_ut.c
Use nvme_ns_construct()/nvme_ns_destruct() for each subtest.
This is what lib/nvme uses when creating/deleting namespaces,
so it makes sense to use the same functions in the unit test.

If we explicitly call memset(), we might hide real problems,
since e.g. nvme_ns_destruct() does not clear all struct members
in struct spdk_nvme_ns.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ia0ffce3d0a29088282e9fc8fc5413b98d1fe7c63
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4264
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-18 07:25:06 +00:00
Tomasz Zawadzki
9b294aeaa6 ut/blob: fix CU_VERSION check for clear linux
On clear linux the # was interpreted as bash comment.
The result was that no file was included and CU_VERSION
was undetected.

Removing the backslash before hash resolves it on clearlinux,
but breaks on fedora 31. Workaround in this patch portable
for both, as strange as it looks.

For future reference:
Fedora 32, Make 4.2.1, gcc 10.2.1, bash 5.0.17
Clearlinux, Make 4.3, gcc 10.1.1, bash 5.0.16

While here, added more specific string detection for
CU_VERSION. Before patch e7a86c42 only CUnit header was scanned,
now other output from CC is. This will prevent accidental
matches in the future.

Fixes #1562

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia4aa9fcbc3ad810755cfecd745c5c1ecd2a70717
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4240
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-17 15:24:50 +00:00
Alexey Marchuk
eb78b90ca8 nvme/rdma: Check that SGL descriptors fit into ICD
The issue happens when SPDK RDMA initiator is connected to a remote
target and this target reports rather small (or zero) ICD and we try
to send several SGL descriptors.
Since SGL descriptors are located in ICD, we should check that their
total length fits into ICD. In other case sending such a command
will cause RDMA errors (local length error)

Change-Id: I8c0e8375dae799bc442ed2fab249cad2c4ccce51
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reported-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4131
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-16 07:58:13 +00:00
Shuhei Matsumoto
071d80f37b lib/nvmf: Add an internal API nvmf_subsystem_set_ana_state()
Add an internal API nvmf_subsystem_set_ana_state() to change the
ANA state of the subsystem listener whose trid matches.

ANA optimized state, ANA non-optimized state, and ANA inaccessible
state are supported. ANA change state is not used and ANA persistent
loss state is not supported.

After changing the ANA state of the subsystem listener, on each poll
group, controllers, whose the subsystem listener match, send ANA
change notice.

Initiators query ANA log page anyway if they receive ANA change
notification. False positive notification should be avoided but is
acceptable.

To avoid any concurrency conflict, simply compare ctrlr->listener and
the passed listener.

It may be better to execute nvmf_subsystem_set_ana_state() on the
subsystem thread but currently the RPC thread adds and removes a
listener to and from the subsystem, respectively, and the subsystem
has been suspended while executing nvmf_subsystem_set_ana_state().
Hence we keep this as a future enhancement.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If1910b79dd33d904114e258ae2c5e868947cdc52
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4079
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
2020-09-16 07:55:28 +00:00
Jim Harris
bd16f57472 blob: switch to bit_pool for tracking used_clusters
We still need to be able to explicitly set specific
bits in the cluster array during initialization and
loading (especially recovery), so we use a bit_array
during load, and then convert it to a bit_pool just
before calling the user's cmopletion callback.

This gives a roughly 300% improvement over baseline
on a benchmark which does continuous resize operations.
The benefit is primarily from saving the lowest free
bit rather than having to always start at bit 0.  We
may be able to further improve this by saving extents
in the bit pool as well, although after this patch,
the benchmark shows other hot spots different from the
bit search.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idb1d75d8348bc50560b1f42d49dbe4d79d024619

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3975
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-15 07:12:44 +00:00
Niklas Cassel
174a5fe140 nvme: add initial namespace types support
Add support for getting the Command Set Identifier for a given namespace.

The SPDK_NVME_CAP_CSS_IOCS feature can be implemented on top of an old NVMe
specification. If the feature is set, retrieve the NS ID Descriptor List
regardless of the NVMe specification version. The quirk is still respected.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I7b257115ecb0d813ba75201c0f48960c7070dcc9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4085
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-11 07:29:51 +00:00
Jim Harris
3c31f6dc03 blob_ut: add create_loop
This test repeatedly creates and deletes blobs to
confirm that the used_md_pages and used_clusters
bit arrays are managed properly.  This test exposed
the bug reported in #1590 which has now been fixed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-ID: If5d7b7f03cc2fb9daac928573c73430b74ea9a51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4038
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-09 07:19:57 +00:00
Shuhei Matsumoto
68f1681771 lib/nvmf: Control I/O and some admin commands according to ANA state
For I/O commands, block them if ANA state is inaccessible, persistent
loss, or change.

For Identify command, clear capacity field (nuse) to 0 if ANA state
is inaccessible or persistent loss.

For Get Features command, block features, error recovery, write
atomicity normal, reservation notification mask, and reservation
persistence if ANA state is inaccessible, persistent loss, or change.

For Get Log Page command, error information page does not return
any data yet, and hence there is no change.

For Set Features command, if ANA state is inaccessible or change,
block the command if NSID is 0xFFFFFFFF or if feature is error recovery,
write atomicity normal, reservation notification mask, or reservation
persistence, or if ANA state is persistent loss, block the command.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I15dd593227e451aa2247c53da42b6acad1757907
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4043
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Shuhei Matsumoto
7bcff376ab lib/nvmf: Add subsystem_listener to struct spdk_nvmf_ctrlr
Find the subsystem listener whose trid matches req->port->trid when
creating a controller, and store it in the controller.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iea343b8d8ae827b554df2245b67aed113469c592
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Seth Howell
58216dd07e lib/nvme: fix mem leak in req submit.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If64c06177605a8f57d87ba22b86fe58ddebd6f7a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3921
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-09-02 07:38:38 +00:00
Jim Harris
150339cc59 cpuset: fix parse_mask
The loop here was counting the bytes in the cpus array,
but the lcores are represented by bits.

While here, add a unit test that exposes this bug and
demonstrates it is now fixed with this patch.

Fixes #1570.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3a1fc48a8085254f41587e3b3d5d732154b90134

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3931
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-27 08:34:07 +00:00
Jim Harris
178291a516 nvme: read all uevents
Currently when the uevent processing code finds a non-uio/vfio
uevent, it just stops its loops and returns.  This means that if
there are a lot of non-uio/vfio uevents, the netlink socket buffer
can build up until its full because only one non-uio/vfio event
gets drained per spdk_nvme_probe() call (which may be very
infrequently).

So modify parse_event so that it does not indicate error when
a non-uio/vfio event is found.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8a40f71ee89d597ce46129eac889fe5b7ef5171

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3876
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-24 07:38:33 +00:00
Changpeng Liu
5adf099eed nvme: don't send Identify NS ID Descriptor List to inactive NS
The specification says it will return INVALID FIELD if the NS
is in inactive state.

Fix issue #1551.

Change-Id: I1b32f023ed665d410f4705e439068699e2b2f8de
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3860
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-21 08:45:31 +00:00
Jim Harris
ed7848f2df blob: handle overlapping open case
We only create one spdk_blob object for a given blob, and just
increase the ref_count if it is opened multiple times.  bs_open_blob
would do the lookup for existing opened blobs.

But if the blob is opened again, before the previous open operation
has completed, we would end up with two spdk_blob objects for the same
blob.

Solution is to do another lookup when the open operation completes.
If we find the blob, free the one we just finished opening and return
the existing one instead.

Also added unit test that failed on the existing code but passes now
with this patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reported-by: Mike Cui
Change-Id: I00c3a913b413deddf06f0b63f7a669efb2b5658f

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3855
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-21 08:24:09 +00:00
Seth Howell
b3bb3a1bbf lib/nvme: change timeout in wait_for_completions to usec
This allows for much more granular control over the timeout.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib23de21e60eec4207c55320579699edf284f4e16
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3794
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-19 07:29:19 +00:00
Shuhei Matsumoto
5af42000c1 lib/iscsi: Make max outstanding R2Ts per connection configurable
By the recent refactoring, we have no static size array for outstanding
R2Ts per connection. It looks that we do not have any critical reason
to prohibit us from making max outstanding R2Ts per connection configurable.

There are some use cases to use large write I/O intensively (e.g. 128KB).
Let such use cases change the value of max R2Ts per connection by their
responsibility to do performance tuning.

Maximum outstanding R2Ts per task are defined both for iSCSI target
and NVMe-TCP target but maximum outstanding R2Ts per connection is
unique for iSCSI target.

The next patch will add the corresponding iSCSI option.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f6fd3c750a9a0a99bcf23064fe43a3389829aa9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-18 07:27:45 +00:00
Shuhei Matsumoto
5b2c76f062 lib/iscsi: Make the max number of read subtasks for large read I/O configurable
For some use case that there is heavy large read I/O, the performance
bottleneck due to MAX_LARGE_DATAIN_PER_CONNECTION was reported.

The following assumes that all I/Os are large read.

Large read primary task whose I/O size is more than
SPDK_BDEV_LARGE_BUF_MAX_SIZE (=64KB) is split into multiple
read subtasks.

spdk_iscsi_globals::MaxQueueDepth limits maximum number of outstanding
read primary tasks, and MAX_LARGE_DATAIN_PER_CONNECTION (=64)
limits maximum number of outstanding read subtasks.

MAX_LARGE_DATAIN_PER_CONNECTION is also used to calculate PDU pool.

To remove the performance bottleneck, change the macro constant
MAX_LARGE_DATAIN_PER_CONNECTION to a global variable
spdk_iscsi_globals::MaxLargeDataInPerConnection.

We don't see any negative side effect if we set
spdk_iscsi_globals::MaxLargeDataInPerConnection to 64.

The use case that reported the performance issue will change the
value of spdk_iscsi_globals::MaxLargeDataInPerConnection by its own
responsibility.

The next patch will add the value of
spdk_iscsi_globals::MaxLargeDataInPerConnection to iSCSI options,
and make it configurable by JSON RPC.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifc30cdb8e00d50f4d3755ff399263cf5d0b681b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-18 07:27:45 +00:00
Shuhei Matsumoto
420b2353ea lib/iscsi: Inform initiator that target has temporary moved to a different address
If the portal group map of the target has a redirect portal,
iscsi_tgt_node_is_moved() fills the buffer by the redirected address
and returns true.

iscsi_op_login_check_target() calls iscsi_tgt_node_is_redirected() before
calling iscsi_tgt_node_access() because login redirection can be
checked before any or after all security check.

If iscsi_tgt_node_is_redirected() returns true, notify login redirection
to the corresponding initiator.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4573a69c0a32eafcfe48080a033c135e127da321
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3221
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-11 08:27:43 +00:00
Shuhei Matsumoto
10d6218444 lib/iscsi: Create portal group as public or private portal group
In SPDK iSCSI target, portal group works almost as identifier of
portal.

To support iSCSI login redirection, we need to have two types of
portal groups, public and private portal groups.

We need portals of public portal groups to redirect to a portal in
a private portal groups at login via temporary login redirection
funciton, and we need to make SendTargets return only portals in
public portal groups.

To do these simply, we mark primary or secondary portal group expicitly
at its creation by this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iccf87a4b9dd1f4a8fbb857a399b8f2dbc7c0b3ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3491
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-08-11 08:27:43 +00:00
Shuhei Matsumoto
1d08c96288 lib/iscsi: Remove unused the param iaddr from iscsi_send_tgts()
Previously we might have any possibility to need iaddr in
iscsi_send_tgts(), but it is highly unlikely now. Let's remove
the param iaddr from iscsi_send_tgts().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ide3405706b727a9d6ebaa92ed83509196ff675da
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3442
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-08-07 15:06:34 +00:00
Shuhei Matsumoto
c15a6180cd lib/iscsi: Add pg_tag to the 2nd parameter of iscsi_conns_request_logout()
The upcoming new feature, iSCSI login redirection will requests
connections whose portal group tag matches to logout asynchronously.
Hence add pg_tag to the second parameter of iscsi_conns_request_logout()
and iscsi_conns_request_logout() checks if conn->pg_tag is equal to
the passed pg_tag.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iaea37f28046396404c5b4faed01d748f2944288c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3160
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-07 15:06:34 +00:00
Seth Howell
0c6979e29b bdev/nvme: add a function for specifying a multipath trid.
This is part of a larger series enabling failover at the bdev
layer for NVMe.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic5c128244699c1a47275145ca7e41aa5f1366259
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3044
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-08-07 15:06:01 +00:00
Seth Howell
d88915f5ff bdev/nvme: rename ctrlr trid parameter to connected_trid
This patch paves the way for introducing a tailq containing
multiple alternative paths to the same controller.

Change-Id: I13d30c12b8e0ce38eae687f9e76740be1d11e4d1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3259
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-08-07 15:06:01 +00:00
Alexey Marchuk
864d93c053 nvmf: Add check for bidirectional xfer type
RDMA target can't handle bidirectional xfer type, in debug build
it throws an assert in nvmf_rdma_setup_wr function. NVMF controller
performs checks od opcodes, but the failure happens before this
check. Add similar validation in TCP transport.

Change-Id: I14400b9c301295c0ae1d35a4330189d38aeee723
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3436
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-04 18:27:45 +00:00
Seth Howell
0d8f86f842 lib/nvme: don't submit request if qpair is disconnected.
This becomes a problem when the qpair is reconnected.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6677b396cf766684a4891ffbee93aa3e4e83374d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-31 08:23:44 +00:00
Anil Veerabhadrappa
03fe6a77c8 nvmf/fc/ut: Add missing function stub
Fix the compilation error by adding spdk_nvmf_request_complete()
  function stub.

Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I564cdb16238d5c45c50895b8b2a512096789ba38
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3523
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-27 20:45:21 +00:00
Michael Haeuptle
62649a7d7b lib/nvmf: Support custom admin cmd handler for abort
The new abort functionality doesn't take custom admin cmd
handlers into account.
This commit allows setting a custom admin cmd handler
for abort that provides the ability to influence the
bdev lookup to which the abort is sent to.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I3a66c6f863f5ee4d89cb2194dffdc6855945fa8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3485
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-24 07:25:47 +00:00
Shuhei Matsumoto
81437ff6ce lib/nvmf: Remove req_to_abort parameter from nvmf_ctrlr_abort_request()
struct spdk_nvmf_request holds req_to_abort and so passing req_to_abort
separately is not really necessary now. The internal API
nvmf_ctrlr_abort_request() was added at the stage of prototyping.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9ef2467d6f92422f044650c62a0777b95c0fc1ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3488
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-24 00:22:24 +00:00
Darek Stojaczyk
45528bfef6 env: add spdk_pci_id->class_id
This follows struct rte_pci_id which had class_id as well.
We'll need it to make some additional DPDK APIs public through
the env abstraction.

Change-Id: I794a6cd6b17e48daf53b48fa5abe3d3dcfeaa403
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3182
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-23 20:48:47 +00:00
Seth Howell
f0be991330 lib/nvmf: complete requests in req_process for inactive qpairs.
There is no reason to continue processing these requests if the
qpair is not still active. We should complete them and free
any resources they are still holding.

Also, not doing so can cause issues with trying to access pointers
in the qpair after they are invalid. See issue #1460.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6e570a576983dfedf726dc4a9a83316209403e00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-22 12:20:55 +00:00
Jacek Kalwas
81717da169 nvmf: use nvme print completion
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: If814edb3c99a7ed9f1800a634f293d3645884457
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3241
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-17 07:20:21 +00:00
Jacek Kalwas
afefd815ca nvmf: use nvme print command
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I22747fb342d2e50f40a3bae4f930957f755a97fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3202
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-17 07:20:21 +00:00
Shuhei Matsumoto
deec1fc790 nvmf/rdma: Abort request whose CID matches if it is outstanding
Call nvmf_ctrlr_abort_request() if the request whose CID matches
is found and its state is executing.

nvmf_rdma_qpair_abort_request() returns immediately if rc is
SPDK_NVMF_REQUEST_EXEC_STATUS_ASYNCHRONOUS, or calls
spdk_nvmf_request_complete() otherwise.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1462a21db7270f3d63f8f293ad4be61d52e74da1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3011
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-07-16 07:29:46 +00:00
Shuhei Matsumoto
b8a87e6af5 nvmf/tcp: Abort request whose CID matches if it is outstanding
Call nvmf_ctrlr_abort_request() if the request whose CID matches
is found and its state is executing.

nvmf_tcp_qpair_abort_request() returns immediately if rc is
SPDK_NVMF_REQUEST_EXEC_STATUS_ASYNCHRONOUS or calls
spdk_nvmf_request_complete() otherwise.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1abceecc211ee79d8ac18a82dc63b13d313a6f27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3008
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
604b4503c4 lib/nvmf: Add nvmf_transport_qpair_abort_request()
State machine is different among NVMe-oF transports and is
encapsulated to the transport neutral NVMe-oF controller and
NVMe-oF qpair.

To implement abort operation for each NVMe-oF transport,
add a function pointer qpair_abort_request to struct spdk_nvmf_transport_ops
and a stub nvmf_transport_qpair_abort_request() to encapsulate
which transport is used.

The following patches will implement qpair_abort_request for each
transport. Each qpair_abort_request() is responsible to call
spdk_nvmf_request_complete() for the abort request.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2beac959ed428c5108cf33691226b7fae5cd24d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3007
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
ee7f961655 lib/iscsi: Unify outstanding_r2t_tasks array and active_r2t_tasks list into the latter
We have no particular requirement to keep both conn->outstanding_r2t_tasks
array and conn->active_r2t_tasks list now.

To improve readability and maintaineability, unify two into the latter,
conn->outstanding_r2t_tasks list. Update unit test accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I25cf7cffbe39ac66e102eb3052340de6ef65c8f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3115
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:39:50 +00:00
Shuhei Matsumoto
d5f3f48b60 lib/iscsi: Fix iscsi_del_transfer_task deletes from both array and tailq
Previously iscsi_del_transfer_task() dequeued the task only from
the array conn->outstanding_r2t_tasks[].

process_non_read_task_completion() had dequeued the task from
the tailq conn->active_r2t_tasks then.

However abort_transfer_task_in_task_mgmt_resp had not dequeued the
task from the tailq conn->active_r2t_tasks then.

This was an apparent bug, and is fixed here. Update unit tests
accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I93f02b2fb670dcee4c32d61c264e3ad5b4f9f43e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3108
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:39:50 +00:00
Seth Howell
f4d0ab7111 Revert "bdev/nvme: add a function for specifying a multipath trid."
This reverts commit 5f13e0f67f.

Reverting this change to allow for a secondary review.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iad208603034bd892f3f248836459005d3c2b5297
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3041
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-08 07:57:11 +00:00
Maciej Szwed
eb05cbd677 pollers: Fix pollers to return correct busy status
Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.

Active pollers should return BUSY status only if they
did any meangful work besides checking some conditions
(e.g. processing requests, do some complicated operations).

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id4636a0997489b129cecfe785592cc97b50992ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-07 07:29:31 +00:00
Ziye Yang
cd8b9455ad test: Enable uring test in CI pool.
This patch is used to enable uring test in some VMs or machines
in the CI pool.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I75c2ad477f5f648289d8dbb344b75b2408d56a38
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3107
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-03 07:31:08 +00:00
Jörg Thalheim
e7a86c42a4 ut/blob: more portable cunit version check
headers can be installed in different directories but `/usr/include`

Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
Change-Id: Ibc4641813f6f1c904560dfe7d05cb9b40109cea7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3134
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-03 07:30:48 +00:00
Alexey Marchuk
e762508854 nvme_rdma: Add check for keyed SGL length
The length of a keyed SGL data block is limited by 3 bytes.
Add a check to fail requests which length exceeds 3 bytes.
In other case we can send an incorrectly formed SGL request with
an invalid or zero length.

Fixes issue #1450

Change-Id: I77cdaff5fbf4be5754a3ac6008b8ccd532ac5905
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3056
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-02 07:21:31 +00:00
Changpeng Liu
b18895d025 UT/scsi_reserve: add Persistent Reserve with SPC2 RESERVE/RELEASE test case
As SPC4R31 5.9.3 "Exceptions to SPC-2 RESERVE and RELEASE behavior" says,
add a SPC2 RESERVE/RELEASE test case when PR is present.

Change-Id: Ibd7e703895cb910637677c6cb28b09e014976e9d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2893
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-25 07:16:08 +00:00
Changpeng Liu
51fe275f10 UT/scsi_reserve: add a typical unit test case for SPC2 RESERVE/RELEASE
Add a test case for SPC2 RESERVE/RELEASE from 2 hosts.

Change-Id: I7b2a6352aa8a90a291f495f059de8920d6cbc11b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2892
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-25 07:16:08 +00:00
Changpeng Liu
6b6a3ff91f scsi: add SPC2 RESERVE/RELEASE support
Some OS(Windows and VMWARE esxi) will send SPC2 RESERVE/RELEASE
commands when formatting the filesystems, but in our previous
implementation of reservation feature, we didn't add such support,
in specification SPC3/4, they all include one section "Exceptions to
SPC-2 RESERVE and RELEASE behavior" feature for compatible support of
SCP2 RESERVE/RELEASE, so we add this support now.

Fix issue #1414.

Change-Id: I65d85ffb3b8f824b0ae4e130f53f01d95735d700
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-25 07:16:08 +00:00
Seth Howell
5f13e0f67f bdev/nvme: add a function for specifying a multipath trid.
This is part of a larger series enabling failover at the bdev
layer for NVMe.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I0d89e1afab8aeaa90237d0ba780d708154f6e3be
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2881
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-22 10:13:29 +00:00
Seth Howell
af9e73302f bdev/nvme: make the controller trid a pointer.
This lends itself to easily changing the trid during failover.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iae21a98de4f9dd1192a50abac0ec7fb0dd8a2e99
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2882
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-22 10:13:29 +00:00
Ben Walker
5584232cce nvmf: Remove new_qpair callback from transport accept function pointer
Transports may now call spdk_nvmf_tgt_new_qpair() instead.

Change-Id: Ib3295c488e22517e82f2051055ae47521d76fe56
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-18 07:29:41 +00:00
Shuhei Matsumoto
315d47da4b lib/nvmf: Clean up abort AER request operation
Include completion operation into nvmf_qpair_abort() and rename
it by nvmf_qpair_abort_aer() for clarification. Update unit test
accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I763cc7d24b979e27e8775f4e69730466a2351bdf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2712
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-10 13:54:40 +00:00
Shuhei Matsumoto
972b3ae3aa lib/bdev: Use not SPDK thread count but core count for mempool cache size
Cache size of mempool is not based on SPDK thread but DPDK thread,
i.e. CPU core. So replace spdk_thread_get_count() by
spdk_env_get_core_count() to determine cache size of mempool in
spdk_bdev_initialize().

Besides, allocate and free stub cores at the corresponding unit tests.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icc4deae84c74820af4de61e991fa3f8683add058
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2767
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-08 09:33:50 +00:00
Shuhei Matsumoto
8d4b319c98 lib/bdev: Change completion status of I/O aborted by reset from FAILED to ABORTED
Using not FAILED but ABORTED for the completion status of the I/Os
aborted by reset will be reasonable and is done in this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8c8181dd33453a330888cffa93505e6fc462f278
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2708
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-08 09:28:27 +00:00
Shuhei Matsumoto
97a5ea5796 lib/bdev: spdk_bdev_abort supports I/O splitting
The last patch ensures that the parent I/O terminate with failure
before continuing splitting process if one of child I/O failed.
This simplifies abort operation for I/O splitting.

Then we can use bdev_abort() and bdev_abort_io() nestedly.

Add necessary unit test together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I562bb6675f1fa380bc53dbe369138317ead66fe0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-06-08 09:28:27 +00:00
Shuhei Matsumoto
089d178abb lib/bdev: Terminate parent I/O with failure immediately if one of child I/O failed
Previously, bdev_io_split_done() had continued splitting process
even if the status became failed. To abort split I/O, this patch
changes bdev_io_split_done() to terminate with failure before
continuing splitting process if the status became failed. Add
necessary unit test together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifd1ea49c22523e8c06fb45ebdcb2c84a57afd2ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2234
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-08 09:28:27 +00:00
Shuhei Matsumoto
fc3e40618c lib/bdev: Add all submitted I/Os to the list including any I/O generated by splitting
This is a preparation to abort split I/Os. To abort any I/O by
bio_cb_arg, they have to be managed by the submitted I/O list.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9459d9d2d3511aad0325dcc20d88610444a4ea04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2231
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-08 09:28:27 +00:00
Shuhei Matsumoto
f048f3d3d3 lib/bdev: spdk_bdev_abort supports the case when QoS is enabled
When the target I/O is controlled by QoS, the abort request is
submitted on the same channel that the target I/O is submitted.

By using this, add a helper function bdev_abort_queued_io(), and
change _bdev_io_submit() to call bdev_abort_queued_io(), and
call _bdev_io_complete_in_submit() with success if bdev_abort_queued_io()
returned true when QoS is enabled on the corresponding channel.
Add necessary unit test together.

Update unit test accordingly, especially, update stub_submit_request()
to abort the matched I/O, update io_during_io_done() because we need to
know not boolean but the exact completion status now, and update
basic_qos() to reset the rate limit to test the abort I/O feature.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I16ca21e7c32cabfdce5d0e5c27a8af8bb00f11c5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2230
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-08 09:28:27 +00:00
Shuhei Matsumoto
7cd20dd3f5 lib/bdev: Add spdk_bdev_abort API
Add spdk_bdev_abort function as a new public API.

This goes all the way down to the bdev driver module and attempts to
abort all I/Os which has bio_cb_arg as its callback argument.

We can separate when only a single I/O has bio_cb_arg and when multiple
I/Os have bio_cb_arg, but unify both by using parent - children I/O
relationship. To avoid confusion, return matched_ios by _bdev_abort() and
store it into split_outstanding by the caller.

Exclude any I/O submitted after this abort command because the same cb_arg
may be used by all I/Os and abort may never complete.

bdev_io needs to have both bio_cb_arg and bio_to_abort because bio_cb_arg
is used to continue abort processing when it is stopped due to the capacity
of bdev_io pool, and bio_to_abort is used to pass it to the underlying
bdev module at submission. Parent I/O is not submitted directly, and is
only used in the generic bdev layer, and parent I/O's bdev_io uses bio_cb_arg.
Hence add bio_cb_arg to bdev structure and add bio_to_abort to abort structure.

In the meantime of abort operation, target I/Os may be completed. Hence
check if the target I/O still exists at completion, and set the completion
status to false only if it still exists.

Upon completion of this, i.e., this returned zero, the status
SPDK_BDEV_IO_STATUS_SUCCESS indicates all I/Os were successfully aborted,
or the status SPDK_BDEV_IO_STATUS_FAILED indicates any I/O was failed to
abort by any reason.

spdk_bdev_abort() does not support aborting abort or reset request
due to the complexity for now.

Following patches will support I/O split case.

Add unit tests together to cover the basic paths.

Besides, ABI compatibility check required us to bump up SO version of
a few libraries or modules. Bump up SO version of blob bdev module simply
because it does not have any out-of-tree consumer, and suppress bumping
up SO version of lvol library because the affected struct spdk_lvol
is not part of public APIs.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I515da688503557615d491bf0bfb36322ce37df08
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-08 09:28:27 +00:00
Ziye Yang
18450e8b82 nvme: solve the spdk_nvme_connect compatibilty issue.
This is used to make spdk_nvme_connect can support
the old library for compatibility.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I49d92fb473c3cbabd8e1240785b920480202eee9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-05 09:04:11 +00:00
Jacek Kalwas
000e6f5b87 nvmf: move cdata subset from transport to ctrlr
Having that transport can decide about particular ctrlr attributes not
globally but per ctrlr.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ia3fb0d4e576cb9f8ce6df75f775e2fd5727d7f48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2757
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-05 09:04:01 +00:00
Wojciech Malikowski
c6dbe0ab0e lib/ftl: Keep tracing functions under define flag
Tracing functions introduce performance overhead
so by default keep them only in debug builds.

Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Change-Id: I08e0a139c3efa87b0f46b7e7a026b0676f3e8634
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
2020-06-05 09:03:31 +00:00
Tomasz Kulasek
550d4867a0 lib/nvme: update io msg producers on ns manage
Change-Id: I20adb92ae4e13e775b5e70617c705afd32e16c9e
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/681
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-06-05 09:01:44 +00:00
Seth Howell
8bef6f0bdf lib/nvme: rdma poll group with shared cq.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ifde29f633f09cccbebfdcde5ab2f96d9590449f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1167
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-04 07:20:16 +00:00
Evgeniy Kochetov
9bcc052bfe ut/sock: Add get/set_opts unit tests for posix sockets
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: If53c1da8501c0c2abf8443643cfc8198c3254811
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-02 09:48:00 +00:00
Evgeniy Kochetov
e9e8735628 ut/sock: Add unit tests for spdk_sock_impl_get/set_opts
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I2025fad3dd19ac706cdbbe28c5684441d2d07374
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/608
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-02 09:48:00 +00:00
Seth Howell
0af754f0f7 lib/thread: remove _spdk prefix.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iab2dba0e7cbbedaeb12f2dd8ac3c8041eb19804c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-01 13:07:30 +00:00
Seth Howell
1a9c19a954 lib/nvme: remove spdk prefix from internal headers.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iccde5860b83217163428ff504cba87a1cf209720
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-01 13:07:30 +00:00
Seth Howell
61d85773f6 lib/nvmf: remove spdk_ and _spdk prefix from functions.
I missed a few files in this library the first time.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2ad55355e6348eaa10384a148dd45deb9f68fc2b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2442
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-01 09:21:14 +00:00
Shuhei Matsumoto
207c959a2c lib/nvmf: Remove inclusion of SPDK event library
Remove inclusion of spdk/event.h and spdk_internal/event.h from
SPDK nvmf library. Their dependency had been removed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibc6c52ab41555d9b29afc3e16c1c3fd0bf5fc63a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2687
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-01 09:20:41 +00:00
Shuhei Matsumoto
6917f6e061 lib/iscsi: Remove inclusion of SPDK event library except shared memory ID for iSCSI top application
iSCSI library still needs to include SPDK event library only for
shared memory ID to allocate shared memory for iSCSI top application.

We will move the code to module library separately later but remove
all other inclusion of SPDK event library in this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I21d77b9cc00c6a1f512546a6a0202c8fa5623bed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2686
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-01 09:20:41 +00:00
Shuhei Matsumoto
7641283387 ut/nvme_pcie: Fix a few assert conditions which had used not == but =
A compiler got warning for these mistakes.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie9772910b6a3cc9d6e45cfae1c19048179d16189
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2710
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-01 09:20:12 +00:00
Alexey Marchuk
88c8d88918 nvme/pci/ut: Initialize hotplug_fd
Change-Id: Ib189ddacfb38bb56aa89a4d81723867eada31833
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2713
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-01 09:19:55 +00:00
Changpeng Liu
3b6622046e nvme/pcie: use 16 bits width access API for PCI_COMMAND register
PCI_COMMAND register with offset 0x4 is 16 bits width.

Change-Id: I86483e58a3ac680efaae6ff7edaadde8efc289f6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2589
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-05-29 14:59:24 +00:00
Jin Yu
d33466d0a9 unit: add a UT for the multi async reqs
The UT can test the received aer reqs number and
check if the reqs are continuous.

Change-Id: Ifa1cb9dd4cdae5526fc80fe6c1b504ad9873d0f8
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2549
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-29 14:59:08 +00:00
Jin Yu
b7cc4dd7dd nvmf: support multi AER events in nvmf target
Add the AER events in nvmf target from one to four.

Change-Id: Ie31988b49d68bdbc28ab2e09c783e681d3017e2b
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2548
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-29 14:59:08 +00:00
Jim Harris
89e47f6014 nvme: create netlink socket during nvme_driver_init
This helps ensure thread safety on creation of the
netlink socket, when probe is called from multiple
threads at once.  It is also a lot more clean - we just
create it once, rather than checking every time probe
is called to see if it has to be created.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I528cedc3ff44de6ea8ecaf6d2389226502ba408e

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2681
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-29 09:26:05 +00:00
Ziye Yang
59e44bcbb3 Revert "sock/uring: use batched manner to get the cqes."
This reverts commit 79215d80c1.

Reason: Find the bugs while using batched. For example, if
we fetch 3 CQEs, A, B, C and put it in a cqes[] array.
Then we leverage io uring cqe seen to handle, Then we handle the
CQE A, then invokes the call back related with A. In A's call back,
it may also call the reap function (sock_uring_group_reap),
then the CQEs will be handled again. Thus the CQEs B and C can be already
handled. Then we will handle B or C again, then it triggers the error.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I67ece3b24e677b88d66d08722b00539693b42e1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2543
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-27 07:35:40 +00:00
Seth Howell
fd36120639 lib/event: remove _spdk prefix from functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ie9aef08d8c46ed9eef883784578fa795e2cd60fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-27 07:35:02 +00:00
Seth Howell
b5d68d5934 lib/blob: remove _spdk prefix from all functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Idb33816e5b66266987845172c27c87667ac0a596
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2437
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-27 07:35:02 +00:00
Sochin Jiang
aaca23fb4b lvol: let thin_provision flag be consistent with Blob
I found this flag weird while debugging a lvol with
thin provisioned Blob, so make it consistent with it's Blob.

Signed-off-by: Sochin Jiang <jiangxiaoqing.sochin@bytedance.com>
Change-Id: Iae555020d5781dfcf4d4f072a47e9573d35b8705
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2435
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-05-22 10:42:28 +00:00
Seth Howell
a3f72b2e5a lib: net, notify, nvme, rocksdb remove spdk_ prefix.
remove only the spdk_ prefix from static functions in
the above libraries.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I59ce032c3312fa73f30c133fd62e603c1eee2859
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2365
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-21 09:19:00 +00:00
Seth Howell
0be5557cad lib: json-nbd remove spdk prefix from static functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Idbf8d37fbac4e3a9eff253095efb2525c9094d94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2364
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-21 09:19:00 +00:00
Seth Howell
a7592dbe13 lib/event: remove spdk prefix from static functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I22ea328f19d7cdb766c12c4c15c188963f971226
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2363
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-21 09:19:00 +00:00
Alexey Marchuk
bbb493ce0c nvmf/rdma: Use RDMA provider API to send WRs
Change-Id: I9bd6956d27716ae95abea0fd78adebaa9b288edb
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1660
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-20 12:03:50 +00:00
Alexey Marchuk
b4a9d7d318 nvme/rdma: Use RDMA provider API to create/destroy qpair
This patch adds use of RDMA provider API to NVMEoF initiator.
Makefiles have been updated with new RDMA lib dependency

Change-Id: Ieaefeb12ee9681d3db2b618c5cf0c54dc52230af
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1657
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-05-20 12:03:50 +00:00
Alexey Marchuk
ea7a4f3c53 nvmf/rdma: Use RDMA provider API to create/destroy qpair
This patch updates NVMF target to use RDMA provider API
to create and destroy qpairs.
Makefiles have been updated with new dependency on RDMA lib

Change-Id: Iae35aea601380f8d1a6453a7fd6115f781e126f5
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-20 12:03:50 +00:00
Seth Howell
937d828f90 module/sock_uring: add a map file and remove spdk prefix.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8c68fc09952d920d6dcf1d8112ba08490e578f84
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2357
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-18 09:48:18 +00:00
Seth Howell
0bc117a470 module/sock_posix: add a map file and remove spdk prefix.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I593c0c1500b2175c4d3dc8ac19289f854895b00f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2356
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-18 09:48:18 +00:00
Seth Howell
30e3f4d900 module/blobfs: remove spdk prefix from static/internal functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2c464bf5a2f220e6f0b8af36dd0dc3283a855df7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2344
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-18 09:48:18 +00:00
Seth Howell
fe5ba3033a module/raid_bdev: remove spdk prefix from static functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8dd833800dbad6c0739ddd13fed546f8b01ec684
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2333
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-05-15 07:58:03 +00:00
Seth Howell
f4653b8a74 module/bdev_pmem: remove spdk prefix from static/internal functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Id8d492068f73682946ab2c3e96393916eb9bb28f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2332
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-05-15 07:58:03 +00:00
Seth Howell
1d2faf6d2b module/bdev_gpt: remove spdk prefix from static/internal functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ibbe4a0fff51ca901aedaca62f424cb67a52100d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-15 07:58:03 +00:00
Kyle Zhang
2187424ab2 lib/vhost: remove dependency on lib/event
In current implementation, lib/vhost assumes that it always runs
together with lib/event, and would call lib/event's functions in
vhost.c. This is not necessary and make program unable to
create/destroy vhost module without init/fini the whole spdk env. It
would cause problems when program runs with vhost and other spdk
components together.

In this patch, we remove the dependency of lib/vhost on lib/event by
adding a global vairable g_vhost_core_mask so that it could handle
core mask by itself.

Signed-off-by: Kyle Zhang <kyle@smartx.com>
Change-Id: I38ceb92ac39b6980955346fda41e968aaead863d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1204
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-13 07:47:01 +00:00
Seth Howell
6dec2087e9 lib/nvmf: remove spdk prefix from static functions in transport.c.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I601d59c5e182aa4a713eb798b3e73fe97106866a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2316
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-12 21:49:03 +00:00
Seth Howell
4de405ab6e lib/nvmf: remove spdk prefix from static functions in tcp.c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If5d29c4236022f949a6f2e44bcd51a6e2a9ea88b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2289
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-12 21:49:03 +00:00
Seth Howell
63c90491e5 lib/nvmf: remove spdk prefix from static functions in subsystem.c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I920c7c850cb1cc0081c2e4c3e5744bd63536786e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2288
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-12 21:49:03 +00:00
Seth Howell
55d8d943a8 lib/nvmf: remove spdk prefix from static functions in rdma.c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I651924850083db8b6a84bdee4c9efd1103551daf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2287
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-05-12 21:49:03 +00:00
Seth Howell
198fd2ce6b lib/nvmf: remove spdk prefix from static functions in ctrlr.c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ibb72c2fb687506831988b09fddad4fc3dae0e4ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-05-12 21:49:03 +00:00
Seth Howell
9cb21ad60a lib/nvmf: remove spdk prefix from functions in nvmf_internal.h
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I9b8d15b52754477c5d120f8d96e0e2c329a32e0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2292
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-12 21:49:03 +00:00
Anil Veerabhadrappa
46f1dd6883 nvmf/fc/ut: Add missing function stub
Fix compilation error by adding function stub for
spdk_nvmf_ctrlr_data_init().

Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: Ia934d82fae6240ccd69c26cfa63ca235f820a813
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2366
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-12 08:11:59 +00:00
Changpeng Liu
87d5d832e7 blobfs: refactor cache_free_buffers()
cache_free_buffers() was only used in the file deletion and unload, and
the file is also freed after that, so here we combine the cache free and
file free together and do the cache free in the cache thread.

Change-Id: I57e9a27c9a6467bcf6c85cd277db3b57e06c98e5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1795
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-05-11 12:34:16 +00:00
Changpeng Liu
c41c4c2bab blobfs: add the file with new cache buffer to cache list in cache thread
Also poll thread in the UT to cover the thread context switch.

Change-Id: I3dc765b66aa707c36eb3913b3e0d2c1c3986e282
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1793
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-05-11 12:34:16 +00:00
Seth Howell
a0878680c0 include/spdk_internal: remove SPDK_LOG_NVME from nvme_tcp.h
This file isn't exclusive to the nvme lib. As such, it shouldn't
use the internal SPDK_LOG_NVME flag.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib6d239ad2e45e58fb97a5ea70b01ce72afa938c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2246
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-11 12:29:07 +00:00
Seth Howell
134590a206 lib/jsonrpc: remove spdk_ prefix from internal functions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ieeb58943536caf46db77ce22509eccb477fa914a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2214
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-11 12:29:07 +00:00
Tomasz Zawadzki
cb7a83f059 unittest.sh: verify additional dependencies by configure flags
There were multiple ways to check if particular UT should be executed.
This script unifies those.

To allow executing unittest.sh as standalone script,
verifying config.h flags was chosen.

FTL is an exception, as it always compiles if built on Linux.
Same condition was kept to execute the UT.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I83b3a58d4574154166af6763e7d1c38d75b8475d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2019
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-05-08 11:30:27 +00:00
Michal Berger
844c8ec383 check_format: Reformat the Bash code in compliance with shfmt
Change-Id: I93e7b9d355870b0528a0ac3382fba1a10a558d45
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1718
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-07 20:52:21 +00:00
Evgeniy Kochetov
4b4b3cca9f nvme/ctrlr: Allow targets not supporting Keep Alive Timer feature ID
NVMe spec defines "Keep Alive Timer" feature ID as optional and there
are targets that do not support this. SPDK fails to connect to such
targets.

This patch allows Get Feature "Keep Alive" target to fail with
INVALID_FIELD status. In this case we just continue with keep alive
timer value stored in controller opts structure. This value is already
communicated to target in CONNECT command.

Fixes #1328

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I52e7ea3cb66073ce6cc168a169989bd179041618
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-07 10:41:31 +00:00
Tomasz Zawadzki
7f007b44a9 lib/blob: clear blobid and md_page on blob_create failure
Blobid and md_page is claimed as first step of blob creation.
If blob creation failed, both should returned to be used by
other blobs.

This caused multiple reports of:
"Metadata page 1 crc mismatch"
when loading blobstore due to md_pages not actually containing
the written out md pages.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I495452c578d879f749281cebf8975eb2c1c7f79a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2057
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-05-07 08:30:19 +00:00
Jim Harris
cfb65ba611 test: add generic unlink wrapper
reduce library uses unlink, but the unit tests need to
override it in a specific way.

But linking unit tests with LTO requires the wrapper
definitions be in objects/libraries listed *after*
the object/library that refers to it.  So we need to
make the unlink wrapper somewhat generic.  We do this
by exporting a string and callback function that the
user can set to enable a user-defined function to be
called when unlink() is called with a specific file
name.

Also revert 3ef6d06 as part of this patch, since we
no longer require the workaround that it implemented.

Fixes issue #1357.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ee4c424ad31fe7d91d7b524ed47aedd279e5b5c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1948
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-06 12:43:57 +00:00
Ziye Yang
79215d80c1 sock/uring: use batched manner to get the cqes.
Purpose: This can improve the performance.

Change-Id: I3f5526ab8716cb0771b5e193afa9a0dbffec5cc0
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1929
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-06 12:43:33 +00:00
Jacek Kalwas
538f1354e0 nvmf: allow to override virtual controller capabilities
Virtual controller capabilities can be overridden on transport
specific layer. The current behavior shall be preserved.

This can be useful to limit or extend the default based on transport
type.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I754f0d957a46f219adc1e55f792e79c7546ddb43
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-28 13:48:17 +00:00
Seth Howell
5d0718528d nvme: implement epoll in the tcp transport.
Change-Id: I6672361baca4969f23259c19b73ed9dbe2f436bd
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/885
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-24 19:38:00 +00:00
Seth Howell
1b818a28b5 lib/nvme: add naive poll_group implementation for rdma.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I55bae6dddc887a95c3e37195fac821de5aa1ed89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/631
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-04-24 16:36:03 +00:00
Seth Howell
fe5e1db68e nvme/tcp: add naive implementation of poll_group api
This implementation simply loops over qpairs calling process_completions.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ia1f59c13444703e00c6b769d378874f48b9ef03e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/627
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-04-24 16:36:03 +00:00
Ziye Yang
94345a0a1a nvme: Add the priority field in struct spdk_nvme_transport_id
Purpose: To set the priority of the NVMe-oF connection especially
for TCP connection.

For example, the previous example can be:

trtype:TCP adrfam:IPv4 traddr:10.67.110.181 trsvcid:4420

With the change, it could be:

trtype:TCP adrfam:IPv4 traddr:10.67.110.181 trsvcid:4420 priority:2

The priority is optional. We try to change
spdk_nvme_transport_id but not in spdk_nvme_ctrlr_opts since
the opts in spdk_nvme_ctrlr_opts will reflect in every nvme ctrlr,
this is short of flexibility.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: I1ba364c714a95f2dbeab2b3fcc832b0222b48a15
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1875
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-24 15:53:34 +00:00
Ziye Yang
b9a7313e2e sock: Add spdk sock opts related three functions.
Purpose: This is used to make users can specify
some options on the socket, e.g., the different priority for the socket.

While creating sockets, the priority needs to be set before connect()
and listen system calls, so better to add one parameter in spdk_sock_opts
which can contain options (e.g., priority) in spdk_sock_listen_ext and
spdk_sock_connect_ext functions.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: I406238e9da7abd69f937b7072535a19124ed0169
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1874
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-24 15:53:34 +00:00
Tomasz Zawadzki
b75e7bd261 ut/idxd: do not run idxd_ut when not compiled
idxd support is disabled by default right now due to being
experimental. Running unittest.sh would cause failure in such case.

With this patch idxd_ut only runs when it was compiled in.
Verifying if the UT was executed is done in post_processing.py,
by verifying every 'run_test' was executed.
Meaning that CI makes sure it is executed.

resolves #1369

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iafbc84839aa9dc62ead8821da8b860e1741e3624
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2012
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-24 15:52:29 +00:00
Tomasz Zawadzki
b31b4229a7 lvol: remove lvol_task structure
Last and only usage of struct lvol_task was removed in
previous patch.

Since it is no longer used, remove the structure itself.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3612106de2ab3a9197245de078c73c53beeff443
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-24 15:45:21 +00:00
Tomasz Zawadzki
4e4c293b9a lvol: do not use task->status
There is no point in setting this value on submission
and changing it on callback.
Since it is based on blob bserrno.

lvol_task is removed in next patch.

This change was motivated by lvol_op_comp() accessing
the task->status pointer on hot path for I/O.
There is no need to do so, so just pass status.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I17461fcf355f771f4de4e6d5ec1900d5042bd3d5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1804
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-24 15:45:21 +00:00
Tomasz Zawadzki
3a574c3c13 ut/lvol: add callbacks and asserts for blob_io_* functions
The UT stubs weren't calling the assigned callbacks
and some weren't assering the passed values.

This will come in handy in next patch.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I86eec58a28e40a93ff06b7e7c715c17d25e28818
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1970
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-24 15:45:21 +00:00
paul luse
58bd4948f0 lib/idxd: add unit test for idxd_wq_config()
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5d505f0642ec3c1297b2772d4ab5c65f51ddfc3a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1823
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-23 15:48:32 +00:00
paul luse
c9391bff85 lib/idxd: add unit test for idxd_group_config()
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I90db6b1a3b37fb76548451a2be284de8d96a44f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1822
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-23 15:48:32 +00:00
paul luse
e333c67bc0 lib/idxd: add unit test for idxd_reset_dev()
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I8a3c50c7ff5cbee1069088a3a06b83f06eb9a85f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1819
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-23 15:48:32 +00:00
paul luse
3075d23318 lib/idxd: add unit test for idxd_wait_cmd()
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ida3185431b526092a0450f2452df665ea0d19abb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1818
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-23 15:48:32 +00:00
paul luse
8e496c5155 lib/idxd: add unit test for spdk_idxd_set_config()
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I8d38491ad7f97310568ec1172df9a9358cfd2a6c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1783
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-23 15:48:32 +00:00
paul luse
f603c27276 lib/idxd: add unit test framework and first unit test
More tests will follow in later patches.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6c45b94cc17219acfae863c774fc7dbc33625c85
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1782
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-23 15:48:32 +00:00
Seth Howell
fc86e792e4 lib/nvme: switch poll group to use connect/disconnect semantics.
This makes more sense within the context of the nvme driver and
helps us avoid the awkward situation of getting a failed_qp callback
on a qpair that simply hasn't been connected.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ibac83c87c514ddcf7bd360af10fab462ae011112
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1734
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-04-22 19:06:26 +00:00
Seth Howell
b874f65743 lib/nvme: disconnect qpairs if they are failed during reset.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I15079cb35d48221bd92b7ca41766148fdb58e668
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1855
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-22 19:06:26 +00:00
Seth Howell
e1c9185005 lib/nvme: always call the transport disconnect function.
The qpair states should be maintained at the generic level.
Always going through the transport disconnect function is
one step in that direction.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I019b2b4a14fe192eff5293f918d633dde2c5400a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1851
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-22 19:06:26 +00:00
Seth Howell
9649ee09fa lib/nvme: rename NVME_QPAIR_DISABLED
This variable really indicates when a qpair is
no longer connected. So NVME_QPAIR_DISCONNECTED is
actually much more accurate.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ia480d94f795bb0d8f5b4eff9f2857d6fe8ea1b34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1850
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-22 19:06:26 +00:00
Seth Howell
bc0180f67f lib/blobfs: remove spdk prefix from private APIs
This should help reduce confusion within libraries about
which APIs are public and which are private.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I7755081b016f269d4761d8cc2ce4bac2835c077d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1703
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-04-22 09:21:55 +00:00
Seth Howell
6569a0ea06 lib/blobfs: add a map file.
Also, increment the library version since some non-public symbols have been removed.

Please note, SPDK_LOG_BLOBFS is included in the map file because it is used by the
blobfs_bdev module.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib05c0b6630951d97b1a5b2931746471eeb9d9630
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1702
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-22 09:21:55 +00:00
Shuhei Matsumoto
1d141f4adb ut/nvme: Fix nightly test warning of test_nvme_completion_poll_cb() by allocating status dynamically
Fix the following warning by allocating status dynamically.

In function ‘nvme_completion_poll_cb’,
    inlined from ‘test_nvme_completion_poll_cb’ at nvme_ut.c:546:2:
/var/jenkins/workspace/unittest-nightly-autotest/spdk/lib/nvme/nvme.c:92:3:
 warning: attempt to free a non-heap object ‘status’ [-Wfree-nonheap-object]
   92 |   free(status);
      |   ^

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I321b88414f431c8c18617d3ee882b8ab851a70fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1940
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-22 09:20:31 +00:00
Ziye Yang
8ad1f4bfa8 lib/sock: remove spdk_sock_set_priority
Since the related feature is already contained in
spdk_sock_listen and spdk_sock_connect functions,
we no longer need this function.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I1eafff0d139fa266a355fbee2bf0fc3947db69fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1876
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-22 09:19:01 +00:00
Shuhei Matsumoto
93d8cab25e lib/iscsi: Remove the "spdk_" prefix from internal APIs declared in iscsi.h
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I81e4934ca38970cfc1707c2d09f6160c9c452045
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1892
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
f7b7195aa0 lib/iscsi: Replace the "spdk_" prefix by "iscsi_" for internal APIs declared in iscsi.h
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1e5a8b567e87ab5693402d164bf73fe22afa45af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1891
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
3570494de2 lib/iscsi: Remove the "spdk_" prefix from internal APIs declared in conn.h
As other small change, function iscsi_conn_pdu_generic_complete()
had been declared in conn.h but defined in iscsi.c. Move the
definition of it to iscsi.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1bd796288036f78a7cba8a1c0af93bd6bc19e9cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1890
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
a587a1072a lib/iscsi: Remove the "spdk_" prefix from internal APIs declared in portal_grp.h
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2c9482c8a89b3dc87f9e4cb55bc298e87472d30c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1888
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
df53885bc5 lib/iscsi: Remove the "spdk_" prefix from internal APIs declared in init_grp.h
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f06612aa6e619371f28b2df435e0966d30da275
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1887
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
6a31372511 lib/iscsi: Remove the "spdk_" prefix from internal APIs declared in tgt_node.h
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I27419445fbd938b56b66a5ba2cc1cde75e7129ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1886
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
356cb2332c lib/iscsi: Remove the "spdk_" prefix from internal APIs declared in task.h
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I746660725f013df93b56ea89d11783ee5c568609
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1884
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
be05a8208d lib/iscsi: Remove the "spdk_" prefix from internal APIs declared in param.h
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8ecdc65f1c0ae88d2de45298db45a8f52e4ed2e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1883
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2020-04-20 07:38:29 +00:00
Shuhei Matsumoto
be1489b9b9 lib/iscsi: Rename iSCSI global data from g_spdk_iscsi to g_iscsi
The following patch will remove the "spdk_" prefix from iSCSI
internal APIs. The iSCSI global data g_spdk_iscsi is also local in
SPDK iSCSI library. Hence rename it by g_iscsi for consistency.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If35e9d58b1388fd725a505ee9be870e414c37ba5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1831
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-20 07:38:29 +00:00
Ben Walker
265a8436f4 nvme: Change mapping semantics of controller memory buffer
Instead of creating an allocator where the driver manages the space,
now, since using the CMB for queues and data has already been
disallowed, just create functions to map and unmap the entire CMB.
The user can manage the space.

Change-Id: I023994deda3b517e14d2ba464c7375bf22b58456
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/785
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-04-16 08:14:18 +00:00
Shuhei Matsumoto
33a2974768 lib/scsi: Remove spdk prefix from private functions
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I89aa5dc10150cff74a420ccf8c3842bd12e84f3c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1825
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:09:14 +00:00
Shuhei Matsumoto
18dec40111 lib/scsi: Remove spdk_ prefix from internal APIs
Remove spdk_ prefix from the name of internal APIs.

Add single underscore as the prefix of the function name to the
private functions if we see any conflict as a result of this change.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If0eabffbdb750db02866c1aa2b29b0c89fab1040
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-04-16 08:09:14 +00:00
Shuhei Matsumoto
799e0a84af lib/scsi: Merge append and execute SCSI task into a single function
Both for SCSI IO task and management task, append and execute
operations bad bbeen separated into different functions.

Append operation was for LUN reset, and separating into two different
functions was for clarification and readability.

LUN reset is sufficiently stable now.

Merging append and execute SCSI task into a single function is good
as API and enables us to do optimization.

For SCSI management task, merge spdk_scsi_lun_append_mgmt_task into
spdk_scsi_lun_execute_mgmt_task() simply.

For SCSI IO task, merge spdk_scsi_lun_append_task into
spdk_scsi_lun_execute_task() and do a small optimization.
The refined spdk_scsi_lun_execute_task() adds the IO task to the
pending list if there is any pending management task, executes all
existing penging IO tasks first and then the IO task if there is any
pending IO task, or executes the IO task directly otherwise.

Update unit test accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I26ffc4f4f62747d8cdecb90690f26cd58a9c17f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1817
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-15 07:40:06 +00:00
Shuhei Matsumoto
2144f0b4c3 lib/scsi: Separate pending task and outstanding (submitted) task into different functions
This is a preparation to the following patches to optimize task
submission and remove the spdk_prefix from internal or private
functions.

To avoid using double underscores as the prefix of the function name,
separate pending task check and outstanding task check into
different functions.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ieda54507bf4c781b60fa05d03cd32ca25948e430
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1826
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-15 07:40:06 +00:00
Jin Yu
ea28cef765 unit: add vhost packed ring test
This unit test shows how the guest and host handle
the shared packed vring.

Change-Id: I4d02a22a225a7945bfe4a2691d917adae86661d6
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/828
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-15 03:55:59 +00:00
Richael Zhuang
a2dbaf7e83 log: spdk_log_dump more characters than required
When the size of input buffer is larger than 16 and not divisible
by 16, the final line will keep part of the characters of the last
line. For example, if I want to dump "spdk dump 16 more chars",the
output is:
00000000 "hexadecimal" spdk dump 16 mor
00000010 "hexadecimal" e charsmp 16 mor
But the correct should be:
00000000 "hexadecimal" spdk dump 16 mor
00000010 "hexadecimal" e chars

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Iaf3f33c5ce68920ada83c59277f89a547e8030d7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1739
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-14 11:34:45 +00:00
Seth Howell
9fe5084860 lib/nvme: when destroying qpairs, abort queued requests.
We should be giving completions for all requests when we destroy a qpair.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I802f5120f2e8289aa825872f8085ac21b5fce0f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1756
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-04-14 11:34:24 +00:00
Jacek Kalwas
55e0ec894f nvme: fix identify active ns
NVMe ctrlr init state machine shall be async whenever possible so it
is not blocking other code from processing. It can result in deadlock
when cmd producer and consumer are sharing the same thread.

This patch is making identify active ns async by introducing new
state to wait for completions.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I346d35bab4733d3941e023602854fdd5b1ef23b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1463
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
2020-04-10 07:08:09 +00:00
Jacek Kalwas
842ae79aa6 nvme: refactor identify active ns
It is a prework for changes related to ctrlr init state machine.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: If289580f65ae27468b659a7ea07a4e4298876e77
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1489
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
2020-04-10 07:08:09 +00:00
Tomasz Zawadzki
e8c42ef37c raid_ut: initialize ch_ctx in test_multi_raid_with_io
Lack of this initialization cause compilation error on release builds.

bdev_raid_ut.c: In function ‘test_multi_raid_with_io’:
bdev_raid_ut.c:2007:3: error: ‘ch_ctx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
 2007 |   verify_io(bdev_io, g_max_base_drives, ch_ctx, pbdev,
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2008 |      g_child_io_status_flag);
      |      ~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2221aa1d78d640439d5876a781a0d173a370e2a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1626
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-09 13:44:21 +00:00
Shuhei Matsumoto
b32411ddaa ut/thread: Test creating and destroying nested I/O channels
This patch creates a test case to simulate nested I/O channels by
generic bdev layer and bdev modules. The nested level of I/O
channels is two at a maximum now but the nested level of the test
case is three.

Besides, any poller registered on the nested I/O channel is not
visible to the upper layer. So include such pollers into the test
case.

By combining thread termination, test the fix for the github issue
is correct.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6a04d753e462cdaba7364678b5118c45b09708ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1390
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
2020-04-09 13:44:14 +00:00
Shuhei Matsumoto
a12aae4c85 lib/thread: Add timeout to wait until exiting thread is exited
Set 5 seconds timeout to wait until exiting thread is exited into
spdk_thread_poll(). After the timeout, collect error log and then
move the thread to exited forcefully.

Add necessary unit test case accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ied8f58a2023a3bbe098530810fd3288bef93c3e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1644
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-09 13:44:14 +00:00
Shuhei Matsumoto
e9aec6746a lib/thread: thread_poll() polls until the exiting thread is exited
Extract _spdk_thread_exit() from spdk_thread_exit() and
_spdk_thread_poll() calls _spdk_thread_exit() if the thread is in
the exiting state. spdk_thread_exit() changes to move the state to
the exiting state. The spdk_thread_poll() loop will end after the
thread moves to the exited state because the caller of
spdk_thread_poll() will check if the thread is in the exited state,
and break the loop if true.

If the user does not call spdk_thread_exit() explicitly, the reactor
has to terminate all existing threads at its shutdown. In this case,
multiple threads may have some dependency to release I/O channels or
unregister pollers. So the reactor has the large two loops, the first
loop calls spdk_thread_exit() on all threads, the second loop calls
spdk_thread_destroy() if exited or spdk_thread_poll() otherwise for
each thread until all threads are destroyed.

Besides, change the return value of spdk_thread_exit() to return
always 0. Keep it for ABI compatibility. Change ERRLOG to INFOLOG
for _spdk_thread_exit() because it is called repeatedly now. Remove
the check of I/O reference count from _spdk_thread_exit() because
_free_thread() cannot free I/O channel. Refine the unit test
accordingly.

Fixes issue #1288.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iee5fb984a96bfac53110fe991dd994ded31dffa4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1423
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-09 13:44:14 +00:00
yidong0635
97e345002b test/nvme_poll_group: Fix uninitialized value in unittest.
Compiling with warning:
nvme_poll_group_ut.c: In function ‘test_spdk_nvme_poll_group_add_remove’:
nvme_poll_group_ut.c:268:8: warning: ‘tgroup’ may be used uninitialized
in this function [-Wmaybe-uninitialized]
  qpair = STAILQ_FIRST(&tgroup->active_qpairs);

tgroup may can't be initialized as:
                if (tmp_tgroup->transport == &t1) {
                        tgroup = tmp_tgroup;
                } else {
                        CU_ASSERT(STAILQ_EMPTY(&tmp_tgroup->active_qpairs));
                }

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Id915e651f73ca3814a7e8d3f95c8793b8b880990
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1738
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-04-09 08:07:50 +00:00
Shuhei Matsumoto
e7ead00b53 lib/thread: Wait until thread_is_exited() is true by thread_poll() after thread_exit()
Following patches will require thread termination so that thread_exit()
will move thread to exiting, thread_poll() will move thread from
exiting to exited, thread_is_exited() will detect the threadis exited,
and then call thread_destroy().

So change all places as a preparation.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6b2e8aee5ed7cd160a88b4c9aaed7d90bd9dac07
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1640
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:43:05 +00:00
Shuhei Matsumoto
c9da0aecdc lib/thread: Ignore spdk_thread_exit() call after moving exited
Previously the caller had to check if thread is not exited when
it calls spdk_thread_exit().

Subsequent patches will change return type of spdk_thread_exit()
to void, and so include the check int spdk_thread_exit() in this
patch.

If spdk_thread_exit() is called when the thread is already exited,
collect INFOLOG and return normally.

This will make the next patch a little easier.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8b94261575e770485b33c0b37e76e770b77b417c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1639
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-08 06:43:05 +00:00
Shuhei Matsumoto
78accbf4e5 lib/event: Count reactor CPU stats (idle/busy tsc)
Following the idea of thread CPU stats, add reactor CPU stats.

Reactor CPU stats accumulates run time of spdk_thread_poll() calls
to idle TSC or busy TSC according to their return codes.

Add necessary unit tests.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I1a1391e79d74387c68f1651a61c8900e4c6faf66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1501
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-08 06:43:05 +00:00
Ben Walker
49826b36cf iscsi: Properly size the receive buffer
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Id736c8a6d9a2cb8bd23017f61eeeccca8a882c03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:42:55 +00:00
Alexey Marchuk
24d61956ab nvme: Init the status object when tracking the req completion
Currently nvme_completion_poll_status object is allocated using
malloc, so it may cotnain some garbage. In some scenarious
nvme_completion_poll_cb can be triggered before we enter
spdk_nvme_wait_for_completion_*. In that case status object
will be freed by nvme_completion_poll_cb if it contains a
garbage in `timed_out` field. Later spdk_nvme_wait_for_completion
will work with already freed memory.
Fix - allocate nvme_completion_poll_status object using
calloc and explicitly zerofy it before usage

Fixes #1292

Change-Id: Iac39653a6cd102471de16e65814f0760bbeda7d9
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-04-07 08:38:58 +00:00
Seth Howell
b9a187977d nvme: add poll group handling to qpair path.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I9116cdcb5bbeb16ee74decee5586bda9a42090aa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/633
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-07 08:38:40 +00:00
Seth Howell
ccf0fd4e39 test/unit: add poll_group unit tests.
Change-Id: Ibcf2d5a3e6d5a27d1ff430ae6a896cb072f151f0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/661
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-07 08:38:40 +00:00
Shuhei Matsumoto
3af1adb8aa ut/compress: Use SPDK thread library
Use SPDK thread library in unit tests for compress bdev. This will
enable us to write unit tests for refactoring.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iac3867cca460b998c9b3953aae5241ffddc3168b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-04-03 06:31:10 +00:00
Shuhei Matsumoto
e92a77b148 ut/compress: Remove unused MOCK for spdk_bdev_io_get_io_channel()
Removing MOCK for spdk_bdev_io_get_io_channel() caused no difference.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic91bff0f163ff5ad607abc8fa4d5eef10215b043
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-04-03 06:31:10 +00:00
Vitaliy Mysak
dcf0ca15c9 ut: make use of CUnit macro to reduce duplications
CUnit provides a helper macro CU_ADD_TEST() that
simplifies usage of CU_add_test() function.
Test name no longer needs to be provided,
as it will be inherited from function name.

This is a follow up to
https://review.spdk.io/gerrit/c/spdk/spdk/+/931

Change-Id: I8078f02e08b14f12328ae022d7090ba13fbd64e4
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1239
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-02 14:50:12 +00:00
Vitaliy Mysak
78b696bca5 ut: simplify error handling at initialization
This patch changes the way that unittests initialize.
The new way is shorter and simpler.
It assumes that error during initialization is a fatal error,
but in our cases it always is.

This patch is a followp up to
https://review.spdk.io/gerrit/c/spdk/spdk/+/930

lvol_ut.c is the only test that skipped because
it runs same tests multiple times which is not allowed
by new method.

Change-Id: I0baf7ad09a35d5fca2dc4a03b4453c12c14f38a7
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1238
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-02 14:50:12 +00:00
Changpeng Liu
68d3bb2de4 nvme: save separate metadata size to nvme request
Previously the SPDK NVMe driver always set PSDT to 01b for hardware SGLs
which is aligned to the Linux NVMe driver, for this case the metadata length
is not required when filling the NVMe command fields.  There is no alignment
nor granularity requirement for Data Blocks for PSDT 01b case.  And if the
drive reported that it needs dword alignment with SGL, for this case, when
using spearate metadata, it needs a length parameter to fill the SGL descriptor.

Change-Id: I56ffaada775fe66de7637dae15b509ee9556e80a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1351
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-04-02 07:02:38 +00:00
Alexey Marchuk
1c5444d623 nvmf: Queue Connect command when subsystem is paused
Currently SPDK rejects Connect command when subsystem is not active.
This change allows to queue Connect command and execute it when
the subsystem goes back to active state. To queue the command we
should know subsystem_poll_group, in current implementation
this poll_group is known only when controller is already created.
To get the poll_group for Connect command we can retrive subsystem
subnqn, find subsystem and get poll_group by subsystem->id.

Increment subsystem_poll_group->io_outstanding even for Connect
cmd in order to prevent subsystem change state during the
connection process. Update spdk_nvmf_request_complete -
decrement io_outstanding for Connect cmd.

Fixes #1256

Change-Id: I724abb911696d7234a9c9d27458eba24739b26fd
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1273
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-01 10:22:25 +00:00
Tomasz Zawadzki
d643b47d9b ut/blob: disable compilation of UT for CUnit other than 2.1-3
This unittest uses CU_add_suite_with_setup_and_teardown()
which was added in CUnit 2.1-3 released over 5 years ago.
This is the most commonly packaged version of CUnit on
not EOL'd distributions.

Unfortunetly there is incorrectly created package on CentOS7.
CUnit-devel-2.1.3-8.el7.x86_64.rpm despite its name
contains CUnit 2.1-2.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5ebf9973b1f3df9ff6cc1f0dc61dcf691096605e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1530
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-01 10:22:17 +00:00
Tomasz Zawadzki
d562d2ae4a ut/lvol: simplify UT by using handy CUnit macros
This kind of changes are now done through out the UTs in SPDK.

Patch here just adds them here as well.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I66fc06bbea038e328ba27bdea58b6e6479eee122
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:10 +00:00
Tomasz Zawadzki
85fc691478 ut/lvol: remove duplicate UTs
lvs_load, lvols_load and lvol_open were already added to the suite.

Commit 2f22884 mistakenly added those three when adding lvol_refcnt.

To remove those UT actually fixes earlier in the series were required,
as incorrect errnos were checked before and this specific order of
UTs set the right errno between them.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7188be49171b6254146c6d144f9a54a689151b60
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1486
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-31 07:57:10 +00:00
Tomasz Zawadzki
4028b671d2 ut/lvol: replace functions with DEFINE_STUB
Simply some of the functions can be replaced with the stub
defines, so they were.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3095855c02c605fe0d66ee720a5025cdd8933873
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1485
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:10 +00:00
Tomasz Zawadzki
764cede766 ut/lvol: consolidate callbacks to op_complete()
There are three types of callbacks in lvol library:
- void(cb_arg,lvs,errno)
- void(cb_arg,lvol,errno)
- void(cb_arg,errno)
First two are used in cases where result of operation is
either lvs or lvol.
Last one is used for any operation that does not produce
either.

There is no need to keep more than one callback function
for last one, since they all are doing the same thing.

lvol_op_complete(), lvol_store_op_complete(), close_cb() and
destroy_cb() are now replaced with op_complete().

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Idd8fcf327ef56ad7d27e2e31cad9ddc44c80e5c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1484
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-31 07:57:10 +00:00
Tomasz Zawadzki
b8645ac8da ut/lvol:use only single global variable to account for errors
There were two variables (g_lvserrno and g_lvolerrno) to
assign and check after operations.
This caused issues when developer mistakenly created callback
that assigns one and then checks the other.

This UT is ran to completion each time and only single operation
is done at once. There is no need to account for both.

This patch removes g_lvolerrno.
Next one will simplify callbacks.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7d16b1913c13de56e1df2f5e36db11f15430979b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:10 +00:00
Seth Howell
24add7e1e4 test/vhost: remove test_vhost.c
This file is not included in any other test files and it isn't compiled.

It appears to be a remnant of when we had distinct vhost_blk and vhost_scsi
unit tests.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I0b29d9e601f9d86b51cf8ba52ec5dc96bd02eef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1310
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-03-31 07:56:15 +00:00
Changpeng Liu
8065ab2c27 nvme/pcie: pass dword aligned requirement based on controller flag
We only set the flag to false when the controller reports SGL supported
and can use byte contiguous buffer.  Also check the data block's alignment
for hardware SGL.

Change-Id: Id936c49823963000d0543fc95fbb6edba3118feb
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1352
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-25 07:54:40 +00:00
Shuhei Matsumoto
2139be1598 lib/thread: spdk_thread_poll() measures run time per call by reading end TSC
Update spdk_thread_poll() to count SPDK thread stats correctly on multiple
SPDK threads per reactor configuration.

spdk_thread_poll() gets start time and reads TSC at end as end time,
and then gets delta between them as run time. Run time is added to idle
time or busy time according to the result of polling.

Reactor overhead is included into the next thread which calls
spdk_thread_poll() now.

spdk_thread_poll() saves the end time to the current thread to use it
as the start time of the next thread.

Unit test framework for this patch and the next patch need to access
thread->tsc_last. In the next patch, reactor will use the end time of
the current thread to the start time of the next thread in reactor_run()
to realize the idea.

Hence add an new API spdk_thread_get_last_tsc(). The corresponding
variable is named as tsc_last and it is good and is aligned with
DPDK (DPDK has used tsc_start and tsc_end as variable name). But
last_tsc will be better as API name because the last TSC value is
easier to understand.

Then add necessary unit test and update the unit test framework.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5e465e9283c032acb427576d0c90f9e1414f2271
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1048
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-25 07:52:53 +00:00
Changpeng Liu
458214e2e7 nvme/opal: remove the revert asynchronous API
The revert asynchronous API doesn't run as the *real* asynchronous
way, because the drive can only support synchronous module and only
1 session is supported.  The reason why we added this API is that
RPC call has the default timeout value here, while the revert may
take over several minutes, the API itself doesn't short the revert
action, so just remove it and use the synchronous API instead.

The revert action will erase all the users data and bring the drive
back to the factory state, it should run in the synchronous mode,
so just remove the asynchronous API and we can increase the timeout
value when using RPC to call this API.

Change-Id: I08a082edea6385e378399423bbb229d05f8bc262
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1232
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-25 07:52:28 +00:00
Tomasz Zawadzki
28a44891f1 lib/blob: add error path on persisting dirty bs
super->clean value signifies if blobstore was unloaded
cleanly.
If it was not, then during bs_load the _spdk_bs_recover()
procedure if called.

Meanwhile bs->clean is always set to 1 after load, causing very first
blob_persist to also re-write super block with the super->clean
set to 0. To signify that md has changed and possibly trigger
the recovery if clean bs unload does not occur.
When the re-write of super block succeeds the bs->clean is set to 0,
because further re-writes of super block are not needed on next
blob persist.

This patch resolves issue when:
1) reading super block fails - execution should backoff, to prevent
writing an empty buffer as super block !
2) writing super->clean = 0 to the super block fails - execution
again should fail, and bs->clean should not be set to 0. It will
cause next persist to attempt re-write again.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia07cc5c6c107310059b50886edb7283c176b9169
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-24 07:39:47 +00:00
Tomasz Zawadzki
a1591e23c4 ut/blob: verify success of spdk_bs_create_snapshot() on power failure
This change verifies that not only it is possible to open
the snapshot, but also the spdk_bs_create_snapshot() reported
success.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifb7fcc91802c838710f1f9be41090057268a5900
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-23 09:37:27 +00:00
Tomasz Zawadzki
2eada474ba ut/blob: recreate bs on each create_snapshot power fail UT
blob_create_snapshot_power_failure() test always attempted
snapshot creation until the snapshot did open.
Which meant that on bs reload after dirty shutdown,
the original blob might be in unrecoverable state.

UT did not verify create call beyond that point.
Next patch will stop after snapshot is possible to be opened,
AND the spdk_bs_create_snapshot() returned success.

This patch makes it easier to recreate same starting
conditions after snapshot did not open.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1901b29a8319b2203a855e7879821ed3850f4917
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-23 09:37:27 +00:00
Tomasz Zawadzki
0f5157377f lib/blob: merge EP of a clone when deleting a snapshot
In general it is not possible to delete snapshot when
there are clones on top of it.
There is special case when there is just a single clone
on top that snapshot.

In such case the clone is 'merged' with snapshot.
Unallocated clusters in clone, are filled with the ones
in snapshot (if allocated there).

Similar behavior should have occurred for extent pages.

This patch adds the implementation for moving EP from
snapshot to clone along with UT.

The UT exposes the issue by allowing delete_blob
to proceed beyond just unrecoverable snapshot blob.

Fixes #1291

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib2824c5737021f8e8d9b533a4cd245c12e6fe9fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1163
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-23 09:37:27 +00:00
Ziye Yang
465f4c1254 sock/uring: Add the async network I/O support for socket
This patch is used to add the async network I/O support in
sock layer. If code is configured with I/O uring, --with-uring,
we can use io uring in Linux (version >=5.4-rc3).

PS: We also make VPP's default priority > uring, because
for the iSCSI or sock test linked with VPP. It tests VPP with
a given address (which is not a special VPP can only open address),
so using uring can also listen those address succefully. And if we make
uring with priority > VPP, actually, VPP will not tested in those cases.

Additionally, the current CI pool is not ready for test, we need
wait for the CI system ready. And I test on my local platform, it works.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie8ee3c8ddf8d2a7264f2b382376733e002816dcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/952
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-20 08:41:28 +00:00
Shuhei Matsumoto
ded6d2c7f2 lib/util: Add spdk_sn32_lt/gt() to compare two numbers based on serial number arithmetic
Previously the macro SN32_LT and SN32_GT had been added to iSCSI
library. But such comparisons may be used in other libraries.
So add two inline helper functions spdk_sn32_lt() and spdk_sn32_gt()
to include/spdk/util.h.

Add unit test for these functions. These functions are located in
header file but math.c is the place if they are located in source file.
Hence add unit test as the one for math.c.

The next patch replaces the macro SN32_LT and SN32_GT by
spdk_sn32_lt() and spdk_sn32_gt() in iSCSI library, respectively.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id3e4d80fea98ad4ae1516e27b9c9e8ec6f37e7a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1346
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-19 08:51:31 +00:00
Evgeniy Kochetov
19453e5b6c ut/nvme: Test Set Num Queues controller intialization step
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I590420fb7d976240a5a924ea7c0204c29d072990
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1272
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-18 08:02:48 +00:00
Evgeniy Kochetov
2ca711694e nvme/ctrlr: Remove Get Num Queues initialization step
NVMe specification in ch.7.6 "Controller Initialization" suggests to
use only Set Features "Number of queues" command and says nothing
about Get Features. All required information is available after Set
Num Queues step.

Fixes #1270

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ide38ba9c7f063f1d6b13bfce4232c588cc906784
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1271
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-18 08:02:48 +00:00
Shuhei Matsumoto
2fa51eeb46 lib/nvmf: Make spdk_nvmf_poll_group_destroy() asynchronous
The next patch will create poll group threads dynamically for
NVMe-oF target, and will need to wait for completion of poll group and
I/O channel destroy. This is a preparation for the next patch.

Add callback function and its argument to spdk_nvmf_poll_group_destroy(),
and to struct spdk_nvmf_poll_group, respectively.

The callback has not only cb_arg but also status as its parameters even
if the next patch always sets the status to zero. The reason is to follow
spdk_nvmf_tgt_destroy's callback and to process any case that the status
is nonzero in future.

spdk_nvmf_poll_group_destroy() sets the passed callback to the passed
poll group.

Then spdk_nvmf_tgt_destroy_poll_group() calls the held callback in the
end.

This change will ensure all pollers are being unregistered and
all I/O channels are being released.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifb854066a5259a6029d55b88de358e3346c63f18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 08:49:00 +00:00
Tomasz Zawadzki
b5ef8c45b5 ut/blob: recreate bs and blobs for each iteration of power fail UT
blob_delete_snapshot_power_failure() test always attempted
snapshot deletion until the snapshot did not open.
Which meant that on bs reload after dirty shutdown,
the snapshot was in unrecoverable state.

UT did not verify delete call beyond that point.
Next patch will stop after snapshot is unrecoverable,
AND the spdk_bs_delete_blob() returned success.

This patch makes it easier to recreate same starting
conditions after snapshot did not load.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7e14e8dadaaef4aa3287ebd376465253466b2362
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 08:23:25 +00:00
Ben Walker
ea65bf612d Revert "nvme/tcp: Change hdr in nvme_tcp_pdu to pointer"
This reverts commit ea5ad0b286.

This code is moving from the nvmf target to the posix sock
layer in this series.

Change-Id: I333bdf325848e726ab82a9e6916e1bbdcd34009c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/446
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-17 08:23:07 +00:00
Konrad Sztyber
02385a64b9 bdev/nvme: asynchronous module_finish
Now that a namespace can be depulated asynchronously now, the NVMe bdev
module should also be finalized asynchronously, after all namespaces and
controllers are deallocated.

Change-Id: Ic082fec8e31e9bd5ee1c698cd8dfca9f248776d3
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-13 08:53:39 +00:00
Konrad Sztyber
9d0f9b330d bdev/nvme: asynchronous namespace depopulation
This patch adds the ability for a namespace to be depopulated
asynchronously.  Currently both regular NVMe namespaces, as well as the
OCSSD ones are depopulated synchronously, but it'll be changed in the
upcoming patches.

The nvme_bdev_ctrlr.ref is now not only tracking the number of bdevs
created on that controller, but also the number of populated namespaces.

Change-Id: I7b112d9b0d41739f3dc7d427e9da340843128c54
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1197
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-13 08:53:39 +00:00
Shuhei Matsumoto
ea863bb0b3 lib/event: Count number of threads per reactor
Add thread_count to struct spdk_reactor to count number of threads
per reactor. This number will be used in the next patch to know
if all threads are idle or not for each reactor to support CPU
power saving.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f7cc5a6b78d85e9f8d0b539c60058c13e282759
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-12 09:04:02 +00:00
Changpeng Liu
c61b70c3d5 nvme/opal: rename spdk_opal_init_dev() and spdk_opal_close() with construct/destruct suffix
Since the OPAL here is just for NVMe device, so we don't need to use dev_handler as common
handler, just rename it to spdk_nvme_ctrlr.  And we don't exit the initialization if
OPAL construnction had a failure.  Also move the timeout initialization to construct().

Change-Id: I11f0aea961eaa3da0c6253eb03d0227f7e7e5f11
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1101
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-11 11:36:08 +00:00
Tomasz Zawadzki
aba9841212 ut/blob: add setup/clean for blob creation
Added new setup/cleanup functions for a suite of
unit tests that use a particular blob through out the
tests.

This simplifies the contents of particular UTs,
and brings forward focus of each UT.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib7d69b90fcbe8e6a0ba303475fcbaeb90fa110a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/945
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
104d4c0a57 ut/blob: add blob deletion at the end of UT
Some of the unit tests were only closing a blob,
without deleting it at the end of unit tests.
This is valid behaviour, but to later simplify
the setup/cleanup - a delete blob was added.

This change was made to unit tests where,
it was not focus of particular UT.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7e338ba6fcb63af26658e8968971a6d56f70807b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1203
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
cc991aad61 ut/blob: simplify blob closing and deletion
Operation of closing and deleting a blob is common
throughout the tests.

This patch creates helper function ut_blob_close_and_delete(),
to do exactly that with the right asserts.
Throughout the UT this function replaces all instances where
blob is closed and deleted, when it is not the subject of
particular unit test (focus is on other functionality).

It will allow to later use it in setup/cleanup functions
for UT suite - similar to bs init.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic56594da8cd4c442b62890d62961b44771059679
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
5df4f50f76 ut/blob: simplify blob creation and opening
Operation of creating a blob (with or w/o opts),
then opening it through out the tests is quite common.

This patch creates helper function ut_blob_create_and_open(),
to do exactly that with the right asserts.
Throughout the UT this function replaces all instances where
blob is created and opened, when it is not the subject of
particular unit test (focus is on other functionality).

It will allow to later use it in setup/cleanup functions
for UT suite - similar to bs init.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia541b855d154503f824bfa3909bcfeac649c5853
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1201
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
95ebed3c9e ut/blob: always use local blobid
Always call API on blobid that was received from blob
create, instead of relying on global one.

Where necessary added declaration and assignment.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3529b4d5e86c1301bb64268029a57873f9f600cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/944
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
114da5a203 ut/blob: always use local blob pointer
Always call API on pointer that was received from blob
open, instead of relying on global one.

Where necessary added declaration and assignment.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I15531111201d071c471f5be79ce18ff508631677
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/943
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
25a01035d2 ut/blob: put setup/clean of tests into common function
A lot of unit tests require only creation of a blobstore
then unload it at the end.
This code can be made common and invoked automatically by
CUnit with CU_add_suite_with_setup_and_teardown().
Those functions are called before and after each test in suite.

This should simplify the unit tests by removing the common
code.

Following patches will add further suites with different
setup/cleanup to reduce common code even more.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I984cde2381f4461e188e06050355024a88ebfca9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
935fbcdf5c ut/blob: do not clear g_bs in helper functions
Next patch in series adds setup/cleanup functions
that use g_bs for its operation.

Particular unit tests should not leave the g_bs
unassigned, if cleanup is supposed to unload it.
It applies to helper functions to reload/dirty_load.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6e022b6c762981c2c70ebf31f4981a17f54f3591
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
a9f76ed204 ut/blob: Do not initialize default bs opts
Some of the unit tests initalized bs opts,
event if they were left unmodified.

There is no need to perform this operation,
as spdk_bs_init() can accept NULL as bs_opts parameter
to just initialize default values.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iacea1fa9d53796b8c359b611092cc63c6bce4094
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1181
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Ziye Yang
9ba4bb22fe lib/nvme_tcp: get the max_sges from the nvme ctrlr.
Add the error print if there is still remaining_size in
order to provide more meaningful debug info.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7b15c9c9a630ea7ecb2d3191b73c9c99f7febf31
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1189
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-11 02:25:12 +00:00
Ben Walker
40529e5d11 nvmf: Add a transport notification when listeners are associated with
subsystems

This is optional and most transports will not implement it.

Change-Id: I51e0f1289b0e61a8bdb9a719e0a2aae51ecb451c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/629
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
2020-03-10 09:12:29 +00:00
Ben Walker
6fc8c8c2fc nvmf: Make spdk_nvmf_subsystem_add_listener asynchronous
In the next patch a new transport call will be added to notify
transports of subsystem->listener associations. The operations the
transports may need to perform there are likely asynchronous, so make
this top level call asynchronous here in preparation.

Change-Id: I7674f56dc3ec0d127ce1026f980d436b4269cb56
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-10 09:12:29 +00:00
Ben Walker
c40f35b764 nvmf: Make spdk_nvmf_tgt_listen synchronous again
This was recently made asynchronous to support virtualized transports.
However, we're moving to add a new call to associated a listener with a
subsystem to transports and the operation that needed to be asynchronous
will actually be performed there. For simplicity, make this synchronous
again.

Change-Id: Ie98136a19c58f0f9bba0d140476de3bbb38e12d7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/881
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-06 10:29:45 +00:00
Tomasz Zawadzki
f422548ea7 ut/blob: reload bs and verify xattr in blob_persist test
This patch adds better check if the xattr was removed.
It will identify potential bugs when first md sync
takes precedence over the second md sync.

As per suggestion in
https://review.spdk.io/gerrit/c/spdk/spdk/+/774/6/test/unit/lib/blob/blob.c/blob_ut.c#7655

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I463f2455614ef11cc1512dc2fb1972ba1f024337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1109
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:28:28 +00:00
Tomasz Zawadzki
682e7c1719 ut/blob: add common function for dirty shutdown
Operation of dirty shutdown of blobstore and reloading
it occurs often enough in UT to provide a common function
to perform it.

Added ut_bs_dirty_load() to facilitate this.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0079dfabd64eaec6495db02fe200be1d6116f0c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:28:28 +00:00
Tomasz Zawadzki
b8fedfa33d ut/blob: add common function to reload blobstore
Operation of unloading and loading blobstore
occurs often enough in UT to provide a common function
to perform it.

Added ut_bs_reload() to facilitate this.

Couple occurences in this patch actually fix
test cases where mistakenly opts were not passed.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib46ed395cd134feaa540e00a334ae861872b3ef4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1091
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:28:28 +00:00
Tomasz Zawadzki
9e6b5aa4c2 ut/blob: always refer to local bs pointer, rather than g_bs
Couple unit tests were still refering to g_bs pointer,
rather than the one local to particular UT.

This patch fixes this.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If343eb561a48c4547d499139d76e1db6b2d7f3bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1090
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:28:28 +00:00
Changpeng Liu
1914de0920 blobfs: start a poller to do cache buffer reclaim regularly
Currently we will free the cache buffers in the file R/W
data path, however we can start a thread to do the cache
buffer reclaim regulary, so that we don't need to free
the cache buffer in the Rocksdb thread context.  By
doing this, we can also remove the global cache buffer
lock in following patches.

Change-Id: Icb0fbc49baecd1e9d86a5fcfe400758dfc3c53a2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:27:45 +00:00
Shuhei Matsumoto
6383602952 lib/scsi: Remove workaround for astyle format check about break-after-return-type
To avoid the starnge formatting, typedef has been used. But this comment
is hard to get the meaning. So stop breaking after return type for this case.

The strange formating is

struct spdk_scsi_dev *
       spdk_scsi_dev_construct(const char *name, const char *bdev_name_list[],
                        int *lun_id_list, int num_luns, uint8_t protocol_id,
                        void (*hotremove_cb)(const struct spdk_scsi_lun *, void *),
                        void *hotremove_ctx)

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I20c30b225b3b96fa9207b4a89e3210c8a97fb1d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-04 10:04:26 +00:00
Artur Paszkiewicz
73763d40ec module/raid: raid5 module
Add raid5 module and unit tests. Use './configure' with option
'--with-raid5' to enable it.

Change-Id: I9f07da8c3567fa65499444899c899adaa2e29550
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-04 10:03:45 +00:00
Ziye Yang
34bff15a19 ftl ut: Fix the compilation issue.
If we did not configure with "--enable-debug". it will
trigger the compliation failure since this function is
not defined.

Change-Id: Ia1795da736be55d4dd1418a2e74edefff7797f11
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1097
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-04 05:02:35 +00:00
Richael Zhuang
ceb601188b test: fix error about unittest_thread uninitialised parameters
When running unittest_thread in fedora29 with memcheck, there is
error "Conditional jump or move depends on uninitialised value(s)"
in for_each_channel_unreg test.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I38abf4c70ff513e39232b83357d571dded8f02c3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-03 07:55:01 +00:00
Konrad Sztyber
336505abbf lib/ftl: use per-io_channel write buffers
Replaced single global write buffer with the per-io_channel write
buffers.  This means that the "rwb" module and all of its references
were removed and replaced with the recently added interfaces.

Change-Id: Idc899d3a4d63a8a2bede1ac26549ed06e9a2e784
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
925cc3b8a9 lib/ftl: pending batches queue
Added queue responsible for keeping track of full, ready to be written
batches.  A batch might be put on this queue in case it'd already been
filled, but could not be written out due to lack of resources or it was
written out, but failed and needs to be resent.

Change-Id: Iba49cd359425300d21b8100f13f17189e99d4c7c
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
87731e1105 lib/ftl: write buffer entry address calculation
Two functions were added:
 - ftl_get_addr_from_entry() returning an address pointing at a given
   entry
 - and it's inverse, ftl_get_entry_from_addr(), returning a write buffer
   entry for a given address

Change-Id: Iea1e2e59f240f50183fdb549280a0f0b722efbef
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
35fdf2dd9e lib/ftl: write buffer batches
This patch adds the ftl_batch strucutre, which describes a single batch
of data to be written to the disk.  It's comprised of multiple write
buffer entries (the actual number depends on the write unit size of the
underyling device).

Additionally, a function responsible for filling out a batch was added.
It iterates over available IO channels and dequeues submitted write
buffer entries until a batch is completed.  The IO channel queue is
shifted, so that each time the function is called a subsequent IO
channel is used first, which guarantees that all channels are treated
fairly.

Change-Id: Ie61d8c6cb51d5c5175540c975447bc27590c5cb4
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/905
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
1e8c9a4de2 lib/ftl: write buffer acquire/release
Added functions responsible for acquiring/releasing write buffer entries
from the per-io_channel caches.  They're also in charge of limiting user
writes by reducing the number of available entries to the user.

Change-Id: I23adfe36065139036f9b72a016e34fbf6eb29230
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/904
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
bfafd4e472 lib/ftl: IO channel array
The IO channel pointers are now stored inside an array of the device
they belong to.  Once write buffer entries are tied to IO channels,
it'll provide a method for dereferencing an entry from its address.

Change-Id: Iaf401525eb0f5af8dc6047a1dc8bae11b56761d7
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Konrad Sztyber
085bfa0d13 lib/ftl: allocate separate memory for IO channels
This is the first patch of a series that replaces single global write
buffer with per-io_channel buffers.  This change is intended to improve
performance for multithreaded workloads.

This patch changes the way the ftl_io_channels are allocated by only
keeping an ftl_io_channel pointer inside spdk_io_channel's context.  It
allows for delaying IO channel destruction, which in turn allows the FTL
to iterate over all exisiting IO channels without locking.

Change-Id: I5e0cab8043a2b5f747e971dd3d65ed2546c8cf26
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-02 10:27:06 +00:00
Shuhei Matsumoto
a4335feb6d lib/event: Support SPDK_THREAD_OP_RESCHED operation in reactor
Add a flag resched to check if reschedule operation is requested
to struct spdk_lw_thread. Add _reactor_resquest_thread_reschedule()
to set the resched flag, and add it to the case SPDK_THREAD_OP_RESCHED
in spdk_reactor_thread_op(), and return true in the case
SPDK_THREAD_OP_RESCHED in spdk_reactor_thread_op_supported().

Then _spdk_reactor_run() checks if the resched flag is true for each
thread. If true, set the resched flag to false, and remove the
thread and call _reactor_schedule_thread(). Add continue to avoid
use-after-free issue for both reschedule and terminate cases.

This idea follows voluntary thread termination and will remove our
worries for all complicated rare cases.

Add unit test case to verify this update.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I656872d32dbb469ae70f771cd0419a77236bfe18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-28 14:37:53 +00:00
Wojciech Malikowski
f93ac08d26 lib/ftl: Merge ftl_thread structure with ftl_dev
Since ftl supports only one core thread ftl_thread
structure is not needed anymore.

Change-Id: I66eda9249d74b3eff0c4473dcd0aba4a6135e296
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/548
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-02-28 08:54:07 +00:00
Michal Berger
c0fb1b93d0 test/unit: [nvmf/ctrlr.c] Increase feature get|set coverage
Add tests for the following features:

  - SPDK_NVME_FEAT_TEMPERATURE_THRESHOLD (4),
  - SPDK_NVME_FEAT_ERROR_RECOVERY (5),

For Temperature Threshold feature the validity of the THSEL and TMPSEL
is covered as described in the NVMe spec (1.4):
	"Figure 279: Temperature Threshold – Command Dword 11"

For the Error Recovery feature the validity of the DULBE is covered
as decribed in the NVME spec (1.4):
	"Figure 280: Error Recovery – Command Dword 11"

Random value is selected for the lsb in the cdw11 (0x42).

Change-Id: Ia57ab4d79439ec315ddc9bbfdad8400aa926062a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-27 10:15:40 +00:00
Shuhei Matsumoto
1368aec870 lib/event: Use spdk_thread_lib_init_ext() to initialize thread library
Remove the prefix "spdk" from spdk_reactor_schedule_thread(), add
spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().

For SPDK_THREAD_OP_NEW, spdk_reactor_thread_op() calls
_reactor_schedule_thread() and spdk_reactor_thread_op_supported()
returns true.

Then replace spdk_thread_lib_init() by spdk_thread_lib_init_ext()
with spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I232a3b2c6bcaf4d86b0dd3cefacd3e47eadda6d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-27 10:14:23 +00:00
Shuhei Matsumoto
d82d69017f lib/thread: Add spdk_thread_lib_init_ext() and generic thread operation function
Add enum spdk_thread_op and two function typedefs spdk_thread_op_fn
and spdk_thread_op_supported_fn.

The first operation type of enum spdk_thread_op is SPDK_THREAD_OP_NEW,
and it is used as an alternative to spdk_new_thread_fn.

Add global variables, g_thread_op_fn and g_thread_op_supported_fn, and
then add spdk_thread_lib_init_ext() to initialize these.
spdk_thread_lib_init() requires both of thread_op_fn and
thread_op_supported_fn are specified or not specified.

spdk_thread_create() calls g_thread_op_fn() with SPDK_THREAD_OP_NEW
if g_new_thread_fn is NULL, g_thread_op_supported_fn is not NULL,
and g_thread_op_supported_fn(SPDK_THREAD_OP_NEW) returns true.

Update unit test to test these addition.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I56db903f62437f6ff3198248ffc5dede396c22bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/967
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-27 10:14:23 +00:00
Jacek Kalwas
fb7129881a nvmf: add const to nqn getters
There is a warning triggered when holding ref to const obj and passing
to these getters.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I2c7b4ea0d325d84d66923fc524273ea44a3a311b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/997
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-26 09:27:11 +00:00
GangCao
c65d64a6b8 vbdev/raid: close the base bdev on its opened thread
With the JSON configure, the base device will be opened on the
same thread (RPC thread) to handle the JSON operation. Later the
virtual device upon the base device can be opened on another
thread. At the time of virtual device destruction, the base
device is also closed at the thread where opening the virtual
device, it is actually different than the original thread where
this base device is opened through the JSON configure.

Add a thread here to record the exact thread where the base
device is opened and then later route it back to handle the base
device close operation.

Change-Id: Ib95e8d190bd87158ae1ecc6698da95ccc4ba9579
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/992
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-26 09:26:49 +00:00
Changpeng Liu
4f7fa18d69 nvme: export internal security send/receive APIs as public APIs
There are synchronous security send/receive APIs defined in nvme.h,
however, we still need the asynchronous APIs so that we can make the
OPAL library can be used in asynchronous way. As the asynchronous APIs
are already defined in nvme_ctrlr_cmd.c, so just export them to public
APIs.

Change-Id: I5646f342a4bf70faad37daa956476f05a1327bcc
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/675
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-26 09:22:15 +00:00
Richael Zhuang
5e1b30bd28 test: fix error about unittest_nvmf uninitialised parameters
When running unittest_nvmf in fedora29 with memcheck, there are
many errors about "Conditional jump or move depends on uninitialised
value(s)". The failed tests are:
in ctrlr_ut:
  test_reservation_notification_log_page
  fused_compare_and_write
in ctrlr_bdev_ut:
  spdk_nvmf_bdev_ctrlr_compare_and_write_cmd

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Ib1e6a744e86876c15ee53206909364e853574dd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/965
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-21 09:39:52 +00:00
Allen Zhu
52a0feb404 bdev/nvme: handle namespace resize event
while the size of namespace is changed,
the resize event will be notified.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Allen Zhu <allenz@mellanox.com>

Change-Id: I5d85f17df898dc21c0ae1eb9f529dcb624a457ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/849
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-21 09:38:42 +00:00
Tomasz Zawadzki
83e8cc3b90 ut/blob: always use local blobstore pointer
Always call API on pointer that was received from blobstore
initialization or load, instead of relying on global one.

Where necessary added declaration, assignment and asserts.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic4198ac984c13a8b77428deaa32ce9864e6750c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/942
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Tomasz Zawadzki
56cec80813 ut/blob: make use of simpler CUnit test addition
CUnit provides a helper macro CU_ADD_TEST() that
simplifies using CU_add_test() function.
Test name no longer needs to be provided,
instead test is named after test function name.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9f5d9e0d8e72ab36b2a5c04e33c336e6c0a1fa6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/931
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Tomasz Zawadzki
b88f0c4f20 ut/blob: simplify error handling for UT
Failure to prepare CUnit for tests is always a fatal error,
and application exits.
Most test managment functions in CUnit set some CU_ErrorCode,
that can be retrived by return code from a function or CU_get_error().

Thus usual structure of UT in SPDK checked every output from every
test managment function.
There is a helper function CU_set_error_action() that when set to
CUEA_ABORT, will abort the application run with proper error return code.
Along with nice error message.

For example if one were to add same UT twice:
"Aborting due to error #32: Test having this name already in suite."
Application return code set to 32.

Using CU_set_error_action(CUEA_ABORT), removes the need for error checking
on test managment functions in CUnit.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ida7de5e040b714509e79957d47a006ee518a42b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Tomasz Zawadzki
a324e6d001 ut/blob: issue to consecutive md syncs
Expands blob_simultaneous_operations to cover case
when second md sync is perfomed without waiting for the first
one to complete.

The blob state is forced in UT, but in normal usage could be
changed by modifying xattr, allocating thin clusters or other.

Related to #1170

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I880d289cdf957d4513c080efdcb7a5d8c6b3b447
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Tomasz Zawadzki
a123ac277e ut/blob: add tests for persiting metadata
This new UT is checking validity of md sync being run
along another md sync with changed metadata.

At this time added case where first md sync is being
interrupted by removal of xattr with second md sync.
This interruption is perfomed at increasing number of
poller executions, until the number is enough to
complete first md sync.

There are two expected states of used_md_pages array,
either with xattr or without. The state is verified
after md syncs.

This UT will be expaned in similar manner by other
operations (than xattr) that casue changes in persisted metadata.

Related to #960

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I251dca92ffc3080d8dc503a7f1ff342aa59adef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:35:27 +00:00
Tomasz Zawadzki
248fbf2aec ut/blob: use poll_thread_times()
Making use of recently added utility to poll exact number of times.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I23a9dc702c899285626522ace8ec77a9ba0f7757
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-21 09:35:27 +00:00
Artur Paszkiewicz
11495b5c63 ut/raid: put bdev_raid.c into 'raid' subdirectory
To make room for other raid module unit tests.

Change-Id: Icf8b8ff0c9051ccb4c7ebd784807370f8be1a8fb
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/853
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-20 14:40:19 +00:00