Commit Graph

896 Commits

Author SHA1 Message Date
Seth Howell
792b36e898 test/rdma_ut: fix valgrind issue.
Recently, we started setting the list of RDMA wr in the parse_sgl
function. This meant that we started using a variable we hadn't before
which was uninitialized in the unit tests which caused a valgrind error.

Change-Id: I3f76ce1dcf95d1d41fe8b3f96e878859036a5031
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443791 (master)
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447450
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-08 22:00:07 +00:00
Tomasz Kulasek
8befeab1b4 test/unit/app_ut: fix potential leak of memory
This patch fixes potential memory leak in spdk_app_parse_args() when
white or blacklist of devices is defined.

Change-Id: Ia586d77c67dbe6c664447f8431e1a7a30d624ae1
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440982 (master)
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-08 19:03:31 +00:00
GangCao
d45c6e54ae QoS: enable rate limit when opening the bdev
There are some cases that virtual bdev open and close
the device and QoS will be disabled at the last close.
In this case, when a new bdev open operation comes again,
the QoS needs to be enabled again.

Change-Id: I792e610f4592bad1cac55c6c55261d4946c6b3e2
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442953 (master)
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447455
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-08 19:03:02 +00:00
Seth Howell
8529ceadfa rdma: adjust I/O unit based on device SGL support
For devices that support fewer SGE elements than our default values, we
need to adjust the I/O unit size so that we don't ever try to submit
more SGLs than we are allowed to.

Change-Id: I316d88459380f28009cc8a3d9357e9c67b08e871
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442776 (master)
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-08 19:00:44 +00:00
Jim Harris
7d1db86f54 iscsi: properly handle partial keys
This includes properly detecting when a key's name
extends past the end of the valid data.

Note that the unit tests were using sizeof() instead
of strlen() since some of the strings contain
NULL characters.  This means that we should be
subtracting one to account for the implicit null
character at the end of the string.  Note that the
iSCSI spec only says that the key/value pair has to
end with a null character - a key/value pair that
is split across two PDUs will not have a NULL character
at the end of the first PDU.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie95d6dd3b9ffa6a3902a31771ac4edb482418cce

Reviewed-on: https://review.gerrithub.io/c/442450
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-29 17:44:12 +00:00
Jim Harris
a95f341361 iscsi: add data_len parameter to spdk_iscsi_parse_param
Since params are parsed directly from the PDU's data
buffer, we need to know the end of the valid data.  Otherwise
previous PDUs that used this same data buffer may have left
non-zero characters just after the end of the text associated
with a LOGIN or TEXT PDU.

Found this bug while debugging an intermittent Calsoft test
failure.  Added a unit test to reproduce the original issue,
which now verifies that it is fixed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic3706639ff6c4f8f344fd58c88ec11e247ea654c
Reviewed-on: https://review.gerrithub.io/c/442449
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-29 17:44:12 +00:00
Seth Howell
9042f3468b test/crypto_ut: change alignment of allocation
This is intended to avoid a crypto failure when we have ASAN disabled.
In order to appease a scan-build error, we have to assert that the
memory address for two g_io_ctx->cry_iov.iov_base is different between
runs of _crypto_operation_complete. However, when we have asan disabled,
two subsequest calls to spdk_dma_malloc yield the same address, so
change the alignment to prevent allocating the same address.

Change-Id: Ia3dfbec5bb16d00bbe7dc136a886e0b9caea71a7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442413
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 08:47:07 +00:00
Shuhei Matsumoto
b78e763c1a string: spdk_strtol to delegate additional error checking
Error check of strtol is left to users of it. But some use cases
of strtol in SPDK do not have enough error check yet.

For example, strtol returns 0 if there were no digits at all.

It should be avoided for each use case to add enough error checking
for strtol.

Hence spdk_strtol and spdk_strtoll do additional error checking
according to the description of manual of strtol.

Besides, there is no use case of negative number now, and to keep
simplicity, spdk_trtol and spdk_strtoll allows only strings that
is positive number or zero.

As a result of this policy, callers of them only have to check if
the return value is not negative.

Subsequent patches will replace atoi to spdk_strtol because atoi
does not have error check.

Change-Id: If3d549970595e53b1141674e47710fe4dd062bc5
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/441626
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 00:10:57 +00:00
Darek Stojaczyk
73d1df6dd6 bdev/crypto: align I/O buffers for encrypt
In patch fbec702944 (bdev/crypto: Set QAT alignment
requirement) we added an alignment requirement for I/O
buffers, but the internally-allocated buffers for
encryption haven't respected it.

We now allocate those buffers with the crypto bdev's
required alignment. It is only required for QAT and we
do it unconditionally, but we don't want to strcmp the
driver name in the hot I/O path just for that - the
code is to be refactored anyway.

Change-Id: I2cbc04408ddc5574f212b63536a05eb73ceba104
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441908
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 21:52:22 +00:00
Wojciech Malikowski
363e48d970 test/unit/ftl: Fix rwb_parallel unit test
This test was causing rare random failures of unit tests

Change-Id: I3031ed9a7f01018f093d9856b719b7604c5e30f1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441875
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jakub Radtke <jakub.radtke@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-24 18:55:56 +00:00
Darek Stojaczyk
d010fe2602 vhost: allow multiple sessions per device
With all the patches in place, we can finally
enable having more than one simultaneous sessions
to a single vhost device.

This patch adds a unique id to the session structure,
similar to the one in a vhost device and also fills in
the implementation holes in foreach_session().

Vhost-NVMe can support only one session per device
and now has an additional check that prevents it from
starting more than one at a time.

Vhost-SCSI also has the same check now since it needs
additional work on the lcore assignment policy. The
check will be removed once the required work is done.

Change-Id: I13a32c7a0eae808e9bec63a7b8c15ec0bc2e36ed
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-24 17:37:03 +00:00
Darek Stojaczyk
6307086792 vhost: untie devices from lcores
It's sessions that are tied with the lcores now.
This makes the vhost devices accessible by any
thread that only locks the global vhost mutex.

The mechanism used for external device events was
refactored to serve for foreach_session() API.

Additionally, since we don't want to handle cases
where the entire vhost device gets removed while
an asynchronous foreach_session chain is pending,
a new per-vdev counter of pending async operations
was added. We'll fail the device removal request
if there are any pending operations. Eventually
we would like the device removal to be asynchronous,
but that's a todo for later.

The external events are still there, although
they only lock the mutex and call the provided
function now.

Change-Id: I20618f9420a9bc04270373469deaad8fb2049c7c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439323
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-24 17:37:03 +00:00
JinYu
b8769cdb08 nvmf: Add the Keep Alive feature
The controller shall treat a Keep Alive Timeout in the same manner
as connection loss. If the Keep Alive feature is in use and the
timer expires, then the controller shall:
1, stop processing commands and set the Controller Fatal Status
(CSTS,CFS) bit to '1';
2, terminate the NVMe Transport connection;
3, break the host to controller association;
A timer poller is added to each subsystem to monitor timeout event.

Change-Id: I001afab8a6764f30c39df37fa96384180d117486
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-24 04:24:11 +00:00
Maciej Szwed
adb39585ef lvol: add option to change default data erase method
Some users require to do write zeroes operation when
erasing data on lvol. Currently the default method is
unmap. This patch adds flag to spdk_rpc_construct_lvol_bdev
call that changes default erase method. This is also a base
implementation for possible future function for erasing
data on lvol bdev.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8964f170b13c2268fe3c18104f7956c32be96040

Reviewed-on: https://review.gerrithub.io/c/441527
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-23 22:25:37 +00:00
Wojciech Malikowski
70b86ec995 ftl: Added unit tests for FTL library
This patch implements unit tests for the following modules:
 * band
 * PPA (Physical Page Address) translations
 * write buffer

Change-Id: Ia7292bd3027347e8a3da77dafe71cde2c016bf38
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431328
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-22 23:22:16 +00:00
GangCao
316cb3b150 QoS/Bdev: add the Read/Write separate bandwidth rate limits
This patch adds the support of read and write separate
bandwidth rate limits control with the configuration file.

Below is the example (in MiB) for the configuration section:

[QoS]
  Limit_Read_BPS Malloc0 100
  Limit_Write_BPS Nvme0n1 200

Change-Id: I0221516ce70c3fbb07b9e80c1c814ed5ba271c88
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/416672
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-22 23:21:43 +00:00
Pawel Wodkowski
1da6e2f5c5 lvol: make unique_id an array
We know how big this should be so no need to dynamically allocate it.

Change-Id: I00ae6cb7c7f525d946d213e0f58cc5fe05bcf932
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441128
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-22 19:45:15 +00:00
tone.zhang
88ddf0f22e UT: fix the sock_ut failure because of the port conflict
In SPDK sock_ut, the port 3260 is tested. It is conflict with the SCSI
Target Daemon (tgtd). If the service is enabled, it makes sock_ut failure.

Suite: sock
  Test: posix_sock .../home/ubuntu/spdk/lib/sock/posix/posix.c: 238:spdk_posix_sock_create: *ERROR*: bind() failed errno = 98
FAILED

The patch changes port 3260 to UT_PORT, and enhance the error log when
socket bind is failed.

Also enhance error log carrying port value in posix.c.

Change-Id: I31e47af49335cdf7eaffa44237860ebc140d2419
Signed-off-by: tone.zhang <tone.zhang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/439983
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 17:28:24 +00:00
Piotr Pelplinski
42dba6047b histograms: add unit tests for public bdev api
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idb725cc9d1477afa458f8d50624996031ddb1b9e
Reviewed-on: https://review.gerrithub.io/c/433644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 17:06:52 +00:00
Ben Walker
c1fbefa1bb thread: Simplify two unit tests with nested poll calls
Now that spdk_thread_poll can be called from within
a thread, some of the unit tests can be a bit simpler.

Change-Id: Ib7b92c3f59862e5c128f5e0cc8a08f65e2b89e93
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441160
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-21 13:54:46 +00:00
Ziye Yang
3c88819bc0 nvmf/tcp: Use the common buffer cache for each polling group
Purpose: To avoid the buffer contention among different
polling groups if there are multiple core configurations
for NVMe-oF tcp transport.

Change-Id: I1c1b0126f3aad28f339ec8bf9355282e08cfa8db
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/440444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-18 19:21:58 +00:00
Tomasz Zawadzki
3bb815ae9f lvol: exposed marking lvol bdev as read only
Added set_read_only_lvol_bdev() RPC that marks
lvol bdev as read only.
This enables to create clones off of a base lvol,
without having to create additional lvol bdev with snapshot.

Change-Id: Ic20bbcd8fbebcef157acce44bfa1da035b0da459
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 19:02:36 +00:00
Seth Howell
e6ddb7df3f rdma: use the new common poll group data buffer cache.
This change is aimed at addressing github issue #555

Change-Id: I5112ac38c59f2f0a17d0c560e7e2f640a11f58a9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 16:57:37 +00:00
Chunyang Hui
01e5610da8 parse: add return value to app_parse callback
Need to check user input and return status of parsing
to prevent app or target from crashing.
Input checking function will be added in the future.

Change-Id: I8167ac13306ae4f81e2cacb80edd9dcf9382c374
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-18 10:08:09 +00:00
Seth Howell
e28605f47a nvmf/transport: move buffer_pool to generic struct.
This is shared between all currently valid transports. Just move it up
to the generic structure. This will make implementing more shared
features on top of this a lot easier.

Change-Id: Ia896edcb7555903ba97adf862bc8d44228df2d36
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-17 19:42:01 +00:00
Shuhei Matsumoto
3f836b036f dif: Fix opposite definition of DIF location parameter from SPDK NVMe API
SPDK NVMe driver and NVMe CLI define the parameter about DIF location as
follows:

If set to 1 and DIF is enabled, then DIF is transferred as the first 8
bytes of metadata. If set to 0 and DIF is enabled, then DIF is transferred
as the last 8 bytes of metadata. Defaults to 0.

This patch fixes the apparent inconsistency.

Change-Id: I481332bb9a6caa488664dbd1fe6039270b9f9663
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/440672
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 17:12:01 +00:00
Ben Walker
9cba82b955 thread: Rename spdk_free_thread to spdk_thread_exit
This name more closely resembles pthread_exit, which is a
closer analogy to how the new threading library works.

Change-Id: I68b04509f3ff8e94b8688804a7e5661155a3ecd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
b1c79d722b thread: Rename spdk_allocate_thread to spdk_thread_create
This mirrors pthread_create, which works more closely
to the new style where SPDK libraries can spawn their
own threads.

Change-Id: Ic524c4c35bcf7c1611e4f261ebb64b98ac5a5a1b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
1068e22da5 thread: spdk_free_thread now takes a thread parameter
Instead of implicitly grabbing the thread from the thread
local variable, make it explicit.

Change-Id: I733fad06181439e12b1e71a4829b84e7b64e2468
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440595
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
6728c4f4c0 thread: Eliminate function pointers in spdk_allocate_thread
These are no longer used by anything.

Change-Id: I0db6bc88e4dc945ff4f64df2ac410e1d00a669c1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-17 11:24:38 +00:00
Ben Walker
11654a6065 test: Use ut_multithread framework in blobfs sync test
Change-Id: I5a15f3af773aad23e0479ad94c4544a2a5f9ea50
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/422470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 11:24:38 +00:00
Jim Harris
42aa5a262b reduce: remove close callback
Let the application figure out how to close the
underlying block device after an init fails or
an initialized/loaded volume is closed/destroyed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id0b9b99c85ed13166ec9a229eaaa0b0eb04289fd

Reviewed-on: https://review.gerrithub.io/c/440573
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-16 22:25:13 +00:00
Jim Harris
c0f3805153 reduce: add spdk_reduce_vol_destroy
This will remove the metadata from the associated backing
device and delete the pm_file associated with the reduce
volume.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I30ffd68e5ba69b31ee1be85418c5b8c592d82e70

Reviewed-on: https://review.gerrithub.io/c/437995
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-16 22:25:13 +00:00
Jim Harris
c887a4f9cf test/reduce: make copy of pmem_map_file path
The value passed comes directly from the spdk_reduce_vol structure,
meaning that we can't reference it after the associated volume
structure has been freed.

We'll need to reference this for the next patch that supports
destroy a compressed volume and unlinking the associated metadata
file.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7ec70972faaec52e405e8d51f4b645c6e623e9dc

Reviewed-on: https://review.gerrithub.io/c/437994
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-16 22:25:13 +00:00
Jim Harris
63ba545f54 reduce: rebuild bit arrays during load
This requires fixing up the unit test pmem emulation
code - we need to copy the 'persistent' buffer into
the newly 'mmaped' buffer in the pmem_map_file stub.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4e474f437922e652a57d0b45b6ef92fc713eb587

Reviewed-on: https://review.gerrithub.io/c/437888
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-16 22:25:13 +00:00
Ben Walker
9bef42f284 test: Use ut_multithread framework in bdev/pmem
Change-Id: Ib9de83db429e4ae6f8e0cc988ca161cebdb27c9c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-15 16:53:12 +00:00
lorneli
94f6d54e2f bdev/gpt: examine my_lba in primary header
Check my_lba in gpt's primary header equals to LBA1.

Set correct my_lba in unit test to pass the test, and update
related header_crc32.

Change-Id: I124a7a883e9304bd3955ce3de6b589596c4195ea
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/439889
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-14 21:26:46 +00:00
Shuhei Matsumoto
8adbd90991 util/string: sprintf_append_realloc to concatenate strings with realloc
Appending string by using sprintf with realloc will be generally
usable and add sprintf_append_realloc() and vsprintf_append_realloc()
to the utility.

These APIs follow realloc about buffer management, i.e., the original
buffer is left untouched if they fail.

Besides, the original buffer is NULL, they are equivalent to
sprintf_alloc() and vsprintf_alloc(), respectively.

Change-Id: I8b69d9640e86e1862ddd3917995bad6f59426b7e
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/436913
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-14 21:22:56 +00:00
Pawel Wodkowski
8687128696 app: add option to read json configuration files during startup
New '--json' long option is added to load JSON configuration file during
startup.

Change-Id: I2a89b7b98b3957e53758dd06686307caf7ec6090
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/430356
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-14 04:48:18 +00:00
Konrad Sztyber
3816dbf89b test/unit: remove unnecessary mock definitions
Added -ffunction-sections / --gc-sections to allow the linker to remove
unused functions when linking test executables. This reduces the number
of required mock definitions to those actually required by tested code.

Change-Id: I22725ce82e22675a7c2d28c8227e586f2cd65023
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/436319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-13 00:48:20 +00:00
Darek Stojaczyk
f82a175706 vhost: separate sessions from devices
Session struct will be now allocated inside the
`new_connection` rte_vhost callback. There can be
still only one connection per device, but this
change brings us one step towards supporting more.

Besides the obvious pointer changes, we'll now also
use the session pointer to check if the connection
actually exists. We used to set device vid to -1
when there was no connection but we no longer have
to do that.

Change-Id: I4d062c0b5f093fef132a6a2c9cc29458cbaad414
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-13 00:46:28 +00:00
wuzhouhui
4086464172 dif, dix: remove unnecessary _iov_iter_cont() check
Because we have called _are_iovs_valid() to ensure that all blocks are
covered by iovec, and we process iovec compactly and orderly, so we
don't need to do _iov_iter_cont() check.

Since all callee are removed, so reomve definition of _iov_iter_cont()
as well.

While here, add an assertion in _iov_iter_advance() to detect
unexpected error.

Change-Id: I6b6077bafbf7ec18f14ee2835d3b99199b5e559b
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/c/439869
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-13 00:01:45 +00:00
Tomasz Kulasek
1c2163b944 util/cpuset: fix internal string buffer size
When all CPUs are set, formatted string overflows str buffer in the
spdk_cpuset structure with '\0'.

It may destroy CPUs bitmap when formatting is used, so additional
integrity checks are performed in UT.

Change-Id: I92ac790b2c215428cbe0ae89ab4b28570ddb9a0d
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440021
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-12 11:19:12 +00:00
Tomasz Kulasek
3d7bf0a6ff util/cpuset: fix and enable ut
One of bugs wasn't catched because unit tests for util/cpuset wasn't
enabled.

This patch fixes bug and enables unit test.

Change-Id: I5a2d7ad76256439e3a13efc8ca51dd4b21a20e0a
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440020
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-12 11:19:12 +00:00
Pawel Wodkowski
6397801009 jsonrpc: fix received JSON buffer overwrite
The receive buffer and JSON values array are shared across all requests
on one connection. If RPC handlers deferre processing response, do the
lazy decode or capture JSON by pointer instead of copying it then
content of the request array might be overwritten by now. As we don't
have any requirement here we must assert that the received request is
valid till response is finished.

Fix this issue by copying request data and work on the copy. This change
also void the need of having JSON RPC 'id' field releasing over 128
bytes from each spdk_jsonrpc_request.

Change-Id: I665be446cbcd8f625e5a73514582efad3021a4ff
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-10 14:31:37 +00:00
Shuhei Matsumoto
ed45352564 dix: Return DIF error information at verification for separate metadata payload
For separate metadata payload, Introduce a struct to collect DIF error
information and pass the reference to the struct to bit-flip error injection
and verification.

This change will make logging of DIF error and comparison between
injection and verification possible for separate metadata payload.

Merging this patch to the previous is possible but add this as a separate patch
to reduce the patch size.

Change-Id: Ifce2130e902bc090cbe205fa8df12559739ced57
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/435097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
65624bd5e4 dix: Generate DIF for separate metadata payload
This patch adds APIs to generate and verify DIF for SGL
payload with separate metadata as byte alignment and granularity.

Change-Id: I4e553c9b6f8c96e576a69115002963d32379d439
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/434150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
045e77c235 dif: Return DIF error information at verification for copied extended LBA payload.
For copied extended LBA payload, Introduce a struct to collect DIF error
information and pass the reference to the struct to bit-flip error injection
and verification.

This change will make logging of DIF error and comparison between
injection and verification possible for copied extended LBA payload.

Merging this patch to the previous is possible but add this as a separate patch
to reduce the patch size.

Change-Id: I9f7ac1332fe63cb206c692d0876c8de41ef9286f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/435227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
bc5507e440 dif: Copy data and generate DIF for extended LBA payload
This patch adds APIs to copy data and generate and verify DIF
for internally created extended LBA payload.

This patch is for read strip and write insert operation to make
DIF transparent to the upper layer.

Change-Id: I025f35571490aa197aac5e7572549e31c2479b41
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/432987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
65e491a8de dif: Return DIF error information by struct at verification
Introduce a struct to collect DIF error information and pass
the reference to the struct to bit-flip error injection and
verification.

This change will make logging of DIF error and comparison between
injection and verification possible.

Change-Id: I43941f1a7f4b0ad1e8ff324f2ac8fad9e195a100
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438023
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-09 19:54:35 +00:00