Commit Graph

1091 Commits

Author SHA1 Message Date
Hailiang Wang
cfd726a23f example/idxd_perf: Adjust the printing data spacing to make it beautiful
Like this:
IDXD_ChanID   Core      Transfers      Bandwidth     Failed     Miscompares
---------------------------------------------------------------------------
         0    0         7006937/s       13 MiB/s   35034688               0
===========================================================================
Total:                  7006937/s       13 MiB/s   35034688               0

Change-Id: I18b0538268ba56d79b77ca6659dc260ee115675c
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16100
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-01-16 15:19:54 +00:00
Mike Gerdts
9e96e331c3 fio_plugin: initalize when create_serialize=0
When the fio plugin is used with reate_serialize set to false, fio calls
the init callback (spdk_fio_init()) before calling the setup callback
(spdk_fio_setup()). When create_serialize is true, spdk_fio_setup() is
called earlier.  Both spdk_fio_setup() and spdk_fio_init() call
functions that require that the SPDK libraries have been initialized.

While it is arguably a bug in SPDK that per-thread initialization
happens before global initialization, it is not terribly difficult to
work around in fio_plugin. The workaround implemented in this patch
splits the start of the plugin's init thread into its own function and
calls it from both spdk_fio_setup() and spdk_fio_init(). The init thread
is started while holding a mutex, and as before there is a guard to be
sure that the init thread is created only once.

Fixes issue #2843

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I20d1863a88b75b416283f9466ee1186d8ef05063
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16053
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-01-11 09:17:16 +00:00
Mike Gerdts
9b754587c7 fio_plugin: bdev calls on spdk thread
This is a rework of commit 7722996dd2.

As the locks in bdev.c transition to spdk_spinlock, callers of
spdk_bdev_get_by_name() and any other function that uses an SPDK
spinlock needs to be executing from an spdk_thread.  This commit reverts
the earlier commit then fixes it with a different approach than was
previously used.

fio plugin functions that need to be on an SPDK thread into two parts:

1. fio API callback: transform args from fio into a spdk_fio_oat_ctx
   structure, then calls spdk_fio_sync_run_oat(). In the case of
   spdk_fio_setup(), the app thread is started in this part.
2. On App Thread (oat) callback: runs on the SPDK app thread via the
   poller spdk_init_thread_poll().

spdk_fio_sync_run_oat() sends a message to the app thread to have the
specified oat callback run. It wakees the poller loop via a condition
variable then waits for the oat callback to signal its completion via a
condition variable in the spdk_fio_oat_ctx.

Fixes issue #2818

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5c82542bb9e9e6b8823e9d1da293050bba336b65
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15866
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2023-01-11 09:17:16 +00:00
GangCao
60fd5c554d examples/bdevperf: check the successful status before dereferencing the pointer
Issue here is that the status could be negative due to NULL pointer and in the
callback, the status is not checked before dereferencing the pointer which is
NULL here.

Change-Id: I7240fa91d1a640556e1f27b8295921c7dd1fb57f
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
2023-01-10 13:11:18 +00:00
Krzysztof Karas
91b8f349fd hotplug: let device insertions/removals take priority
When running io_loop() let insertions and removals counters
take precedence before timeouts. This will let us determine
the cause of premature io_loop() exit in case of timeout.

Consider two scenarios:
First:
 1. Hotplug app records enough insertions/removals
 2. During "if (now > tsc_end)" check it also notices that
    it should terminate due to time constraint
 3. Hotplug exits using the timeout condition, because
    enough time has passed and no error is printed at
    this point.

Second:
 1. Hotplug app did not record enough insertions/removals
    during app runtime
 2. During "if (now > tsc_end)" check it also notices that
    it should terminate due to time constraint
 3. Hotplug exits using the timeout condition, because
    enough time has passed and no error is printed at
    this point.

After prioritizing the check on counters we avoid the first
scenario and timeout will be checked after the counters
were confirmed to be less than expected value, allowing us
to determine the cause of failure earlier.

Additionally added an errorlog to the body of the if checking
for timeout to signal this failure.

First part of the series fixing #2201.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I3f6f9c5c95e82e0a003419fcf181d858ffbd94dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2023-01-09 11:16:50 +00:00
GangCao
0ab8393c60 examples/bdev/fio_plugin: add the RPC listen support
With this support, user could send the RPC commands to FIO
to query the IO statistics like output of below commands:

root@storage35:/spdk# ./scripts/iostat.py
cpu_stat:  user_stat  nice_stat  system_stat  iowait_stat  steal_stat  idle_stat
           3.18%      0.00%      0.35%        0.00%        0.00%       96.46%

Device   tps   KB_read/s  KB_wrtn/s  KB_dscd/s  KB_read  KB_wrtn      KB_dscd
TestPT   0.01  0.00       0.04       0.00       36.00    499608.00    0.00
Malloc1  0.61  0.00       2.42       0.00       36.00    31012820.00  0.00
Malloc3  0.01  0.00       0.04       0.00       36.00    499608.00    0.00

Change-Id: I694119313fb92cf9798c6897e61f788c2ac1f013
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15984
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2023-01-06 15:58:47 +00:00
GangCao
592fc6f410 examples/nvme_manage: check the valid address parse
Change-Id: I40daaae514902854c5d546388789e88248fe6869
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16078
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2023-01-05 23:26:02 +00:00
Michael Haeuptle
031ba90fe1 perf: Support setting IPv4 type of service (TOS)
New option --transport-tos to allow setting TOS value for RDMA transport.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I7747fac5c6641dfd18356314e8209bf5f3f35b7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2023-01-05 19:54:53 +00:00
GangCao
ed4b89aaab examples/bdevperf: reset the performance period for next run
In the case of running by below commands:

./examples/bdev/bdevperf/bdevperf.py perform_tests

This value needs to be reset in each run, otherwise, it shows
wrong IOPS in the periodical way with -S parameter.

Change-Id: Ib8de7611aa06675d4a16fc3aa130d99b508b7605
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-12-21 09:34:12 +00:00
GangCao
f042d6bac4 examples/accel_perf: correct the unlock operation
Change-Id: I43e54ceae1184483747dff70d224a7aa2b24a6fd
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16006
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
2022-12-20 09:19:28 +00:00
GangCao
857e8cae85 examples/sock: close the right sock object
Change-Id: If26e4ace04e2ab6a950ebd270a46805ed8ca973d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15956
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-12-16 09:26:11 +00:00
Alexey Marchuk
e58885f949 bdevperf: Limit queue depth for verify job with big IO
By design verify payload, uses a bit array to find
an offset of IO request. The bit array's size is
calculated as bdev_num_blocks / (io_size/block_size),
if bdev is small, queue depth requested by the user
might be bigger than the bit array size and in that
case bdevperf won't find a free bit for IO request
offset.

To fix that issue, limit queue_depth of such bdevs
by "size_in_ios" value

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I3117f5af7ae3ea18219c25982f33db936dd24c0b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15777
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-12-14 08:02:55 +00:00
Alexey Marchuk
893aaacccb bdevperf: Check current_queue_depth when job starts draining
During debug of some issue, it was found that
current_queue_depth might be 0 when job timer expires,
in that case we won't trigger the job end flow and
bdevperf may hang.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ia49ce6905d329f3ef40216c277bf095782ac9b2d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15776
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-12-14 08:02:55 +00:00
Michal Berger
483d34ad03 bdev/bdevperf: Fix spdk_bdev_channel_get_histogram() arguments
Don't pass bdev argument as it was completely removed from the
function via:

https://review.spdk.io/gerrit/c/spdk/spdk/+/15837

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Icbc1265a516e9fd21820b2f433ced4d4522b46b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15893
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-12-13 11:51:25 +00:00
GangCao
5ba9f929a6 examples/nvmf: explicitly set the initial value for local variables
Change-Id: Id1638718328d911a4d19aeb00ae9a0d7c650262e
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-12-13 09:26:18 +00:00
Richael Zhuang
045c781d47 bdevperf: use SPDK_SEC_TO_USEC for unit conversion
Use SPDK_SEC_TO_USEC instead of '1000 * 1000' or '1000000' for
unit conversion.

Change-Id: Ie9fb016b51ef3346ec640297941408f6c643b9b8
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15784
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-12-13 09:24:53 +00:00
Richael Zhuang
723dd06eb8 bdevperf: support latency info in bdevperf output
Get latency info from bdev histogram and print latency info in
bdevperf output.
print min/max/average latency info by default;
print summarized histogram info if run bdevperf with '-l';
print detailed histogram info if run bdevperf with '-ll'.

Change-Id: I3261574a12c038acde81a58727e9942c0f8c57b2
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14640
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-12-13 09:24:53 +00:00
Michal Berger
3f912cf0e9 misc: Fix spelling mistakes
Found with misspell-fixer.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: If062df0189d92e4fb2da3f055fb981909780dc04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15207
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-12-09 08:16:18 +00:00
wanghailiangx
95fd6cb455 example/nvme: Adjust logic lines about if (current_queue_depth > 0)
The original code is logically correct because
spdk_nvme_qpair_process_completions() may decrease
ns_ctx->current_queue_depth. But I still recommend doing
an optimization so users can easily understand the code.
Prioritize code cleanup more than one additional polling.

Change-Id: I3613b1b4e294661fd35b7e5bebdb32e1c8352270
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14332
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-12-07 08:42:55 +00:00
Mike Gerdts
7722996dd2 fio_plugin: spdk_bdev_get_by_name on spdk thread
As the locks in bdev.c transition to spdk_spinlock, callers of
spdk_bdev_get_by_name() and any other function that uses an SPDK
spinlock needs to be executing from an spdk_thread. This commit
initializes threads in the fio plugin in slightly different places to
accommodate this requirement.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I363ec544e58cb3540414305bc20e2d44bf225599
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15535
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-12-06 21:20:17 +00:00
Kozlowski Mateusz
2decb876cd examples/vmd: Add nvme library to VMD example apps
NVMe driver is necessary during spdk_vmd_init as vmd_init_end_device
won't find the nvme driver during spdk_pci_nvme_get_driver call,
resulting in segfault during spdk_pci_hook_device.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I8fc3396b467e34ec49e39075dca05eeb78f9c362
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15593
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-12-05 09:35:14 +00:00
Michal Berger
588dfe314b Add SPDX header to various files
They were missed by the initial set of patches which introduced this
header as a mandatory one across different types of files.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I3f9b37d41298c843e1648e72fe8593768ccd37e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15423
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-29 08:27:51 +00:00
Ben Walker
27e85f520b accel_perf: Add compress and decompress support
Compression and decompression both start with an uncompressed file,
provided using the new -l command line option. The file is then
compressed in chunks according to the specified block size. Each
operation works on one chunk.

Change-Id: Ia7d3853627d938f73e6aa3ee09fccd11d9bca706
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Paul Luse <paul.e.luser@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14681
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-11-28 09:45:27 +00:00
GangCao
f43678c18e examples/perf: close the fd for URING or AIO
Change-Id: Ieb120ad2d0e4de965b85ce12ac421fe0906dada2
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15660
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-28 09:44:55 +00:00
GangCao
ef9168dcaa examples/bdevperf: fix potential memory leak of allocated config
Change-Id: Idd9d9bec5df0fbb74bb1f2a8af61c408a150d1aa
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15536
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:53 +00:00
Jim Harris
081b190b5f bdevperf: call spdk_thread_exit on job threads
Note: we can only exit a job thread after all
jobs are done executing.  This is because some bdevs
like Ceph send messages to a main thread, so we need
to make sure a main thread doesn't exit before another
thread that is sending messages to it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa51f13f3bc659f6eda7fcefff82c127e095ce29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15516
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
dcb296a32b bdevperf: add bdevperf_job_free() function
Introduce a helper function to properly free everything
associated with a job structure, and then use that
in all of the various error paths to reduce the
number of changes required if/when new strings are
added to the job structure.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I05280371326ce9582e8452a7753a7990072f25b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15506
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
7aa2cc29c0 bdevperf: simplify freeing of 'outstanding' bit array
spdk_bit_array_free(NULL) is fine, so we can avoid the
check.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I989797d5dce0339efb18a7e47e8d974d4ac5abf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15505
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
Jim Harris
0e7821e99d accel_perf: call spdk_thread_exit()
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id93c64a5eda13b21d36bc2006f078a7df76d0e56
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15502
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-11-23 08:22:04 +00:00
wanghailiangx
0c5d34addd test/nvmf: avoid extra error of abort with cbarg when IO queue size 128
Lots of extra error bellow in CI when run abort.sh with IO queue size 128:
nvme_qpair.c: 609:nvme_qpair_abort_queued_reqs_with_cbarg: *ERROR*: aborting queued i/o
Consider setting disable_error_logging = true to avoid this error.

Change-Id: Ic9e2b0ba11a0aa7d66c5da61a2b4fe7ba9fae5a7
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14284
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2022-11-17 08:55:33 +00:00
Krzysztof Karas
29784f35cd bdevperf: promote bdevperf from test to example application
bdevperf application is utilized in multiple test scenarios
and most prominently in SPDK performance reports.
Similar to perf and fio_plugins, it is used to measure performance.
It is intended to represent the expected behavior of users
application that use SPDK bdev layer.

Applications under --enable-tests are intended for specific test
scenarios and user interaction is usually via a test scripts.
Meanwhile --enable-examples are used more broadly to teach and
show usage of SPDK libraries.

As an added benefit with this patch, its possible to compile
bdevperf without need to satisfy additional requirements to
compile all the test applications.

Change-Id: I9aaec1f9d729a1cdee89b5fdc365d61c19b03f82
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14558
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2022-11-16 09:52:41 +00:00
paul luse
7fc8a7dd45 add Intel copyright notices (and/or SPDX lic) where missing
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line.  Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: If1bf46b4e1902585c318dc35b1673f980366076e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-11-10 08:28:53 +00:00
paul luse
a6dbe3721e update Intel copyright notices
per Intel policy to include file commit date using git cmd
below.  The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc).  Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line.  Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-11-10 08:28:53 +00:00
GangCao
03b1b86078 test/bdevperf: set a random seed for rand rw testing
To fix issue: #2726

And also fix the examples/nvme/perf tool.

The srand() only needs to be called once to set the seed of
futher more calls of rand().

Change-Id: I41ab3a46593513516ad11ea7a5b8960b449e9867
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15108
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-10-25 07:11:30 +00:00
GangCao
eab96073cb test/bdevperf: set a random seed for rand rw testing
To fix issue: #2726

And also fix the examples/nvme/perf tool.

Change-Id: I4d1d362b7c1096d27a2d431851462d7bd161df88
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
2022-10-21 07:16:50 +00:00
Jacek Kalwas
74649fb45a perf: add number_ios option
normally IOs are submitted until exhausted allocated time

new option is additional exit ciriteria

provided value represents number of I/O to perform per thread on each
namespace, when this condition detected app will print report and exit

new io_submitted stat is introduced to prevent submiting more cmds
than specified by number_ios and initiate draining

it is similar to number_ios fio option

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Icd188c4e3f17abf277d972277834ddf230c38503
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14365
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-10-03 08:47:08 +00:00
Artur Paszkiewicz
65e08d1265 examples/hello_bdev: support write_unit_size
Change-Id: Ifd9b73560d1f47d545825b69c20a1f02204596a9
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14713
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-09-29 22:52:45 +00:00
Indraneel M
5eafc3a279 bdev/uring: Proper handling for conventional zones
Identify and properly handle conventional zones (in smr drives) by using
zone type and WP state. Bdevs supporting zoned devices(like uring, nvme and
vbdev_zone_block) now update the zone type information. As a result, the
fio plugin now uses this info instead of hard coding the zone type.
Also adds new WP state(ZONE_STATE_NOT_WP) for handling zones w/o WP.

Signed-off-by: Indraneel M <Indraneel.Mukherjee@wdc.com>
Change-Id: If031e0742d68c55c35e95ddc33d478939bbd52fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14572
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2022-09-27 19:40:44 +00:00
Krzysztof Karas
ce29e0131f hotplug: use RPC instead of reading hotplug logs
A new RPC perform_tests has been added.
This request will help us avoid reading hotplug generated
logs, and instead we can stop hotplug right before
it begins generating IO, and resume it when the test
scripts are ready.

Additionally a new command line option has been added
to the hotplug application "--wait-for-rpc", which
indicates that hotplug should wait for
perform_tests RPC before starting its IO.

Change-Id: I71ca148201854ac155cc2a61171a4fb5fc427a19
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13962
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-23 12:54:44 +00:00
paul luse
850cd90082 accel/idxd/iaa: Convert to use iovecs
In prep for upcoming iovec based compression/decompression patches.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I413493f764bead9e56266e488b74f8bca979e225
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14633
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-23 00:10:08 +00:00
paul luse
28886ac352 lib/accel: rename iovec elements with src prefix
In prep for adding both src and dst iovec support for compression.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I704b8d2bd459de03deb7f8ee45d76261910a3727
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13746
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-09-23 00:10:08 +00:00
Ben Walker
712e8cb7ef accel: Refer to plugins as 'modules' instead of 'engines'
This is consistent with the use of terms in other parts of SPDK and fits
with the code living under module/

Change-Id: If182f7cf2d160d57443a1b5f24e0065f191b59b2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-09-21 08:17:48 +00:00
Jacek Kalwas
b9717f0109 perf: change qdepth from int to uint32_t
this is due to further change and compiler warning related to
comparison of variables with different signedness

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ia1e5acd85fce27fc53f63597b0e44753b86a7801
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-20 00:46:38 +00:00
Tomasz Zawadzki
dcd1fb796c example/perf: move allocating ns_worker
Next patch in series will add another caller for
the allocate_ns_worker.

Change-Id: I60281b519265b27d661c2b1ca94e09dc68dbbf57
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:13:09 +00:00
Tomasz Zawadzki
b6e9e1abb4 example/perf: comment the ns to worker association
This documents the logic below and motivation behind
(70db0e1)nvme/perf: Add support for multiple cores per device.

Change-Id: I949b794355f871c0f995b946622dbe77295d922b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14413
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-19 13:13:09 +00:00
wanghailiangx
69b4e8b17f test/nvme: Replace -s with -o for "io size in bytes"
For most of apps, -o is for "io size in bytes",
-s("io size in bytes") was changed to -o a long time ago.
-s is now for "memory size in MB for DPDK".

Change-Id: I0977d4a7bc30a4f2e135707ff6e77b6254240fb9
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14533
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-19 13:11:55 +00:00
wanghailiangx
809ae055b4 usage: Add ] for [-G enable debug logging
Change-Id: I02944e2c40c9a5d288d20b53548371d5e7b97069
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14334
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-09-06 07:16:32 +00:00
Ben Walker
34c48f1b3b accel: Do not refer to the "framework" as "engine"
The word engine was both used (interchangeably with module) to refer to
the things that plug into the framework and to the framework itself.
This patch eliminates all use of the word engine that meant the
framework. It leaves uses of the word that meant "module".

Change-Id: I6b9b50e2f045ac39f2a74d0152ee8d6269be4bd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13918
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
2022-09-06 07:16:17 +00:00
Richael Zhuang
8c4f6f49fe perf: allow to set zerocopy_threshold
add --zerocopy_threshold option to allow data sent with or without
MSG_ZEROCOPY flag according to data size in perf test. This option
can be set when specifying --zerocopy-threshold-sock-impl impl.

Change-Id: Ic187aadba38287bc8f2f082319e26881a13a6766
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-09-01 08:27:01 +00:00
Ben Walker
f1cba369ed identify: Report the maximum number of I/O queues per controller
This is the value after negotiation with the host.

Change-Id: Ifec843c7c2ff7d88ee3ccf26b942fcd084efda55
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13798
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-08-30 07:18:35 +00:00