Commit Graph

2681 Commits

Author SHA1 Message Date
Dariusz Stojaczyk
679e2831bd vhost: added rpc commands to remove vhost controllers and devices
Added new rpc commands together with underlying vhost API and tests.

Change-Id: Ib9c6a530d0909193ea5115aaac4920c44f39613c
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-05-18 01:01:46 -07:00
Pawel Wodkowski
82d26c4f20 vhost: track VFIO mappings
If more than one controller is added to guest the same memory table is
set multiple times making VFIO registration/unregistration failures

Change-Id: Ib55c38e292495e549c070beb0234e73f269e63d5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2017-05-18 00:47:55 -07:00
Jim Harris
f2c8d0dc11 test/iscsi_tgt: exit rbd.sh if CEPH_DIR not defined
This allows the iscsi_tgt tests to be run more easily
in isolation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1fa87e8b02f6e6ed62c75f07471d93ce84da30c2
2017-05-17 15:41:56 -07:00
Paul Luse
b499ae64ad doc: update readme
Clarrifications for DPDK submodule

Change-Id: I7374c1f904f1c5c46c46c299af72816d96a770ff
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
2017-05-17 13:42:32 -07:00
Dariusz Stojaczyk
709d5453a3 vhost: obey VIRTIO_F_NOTIFY_ON_EMPTY
It'll trigger interrupts even if VRING_AVAIL_F_NO_INTERRUPT is present.
Also delegate notifying check to separate function vq_should_notify. It
will be expanded even more once we implement VIRTIO_RING_F_EVENT_IDX
support.

Change-Id: Ic41fd18c9c005c6266b109b0eda4a6dfaef5f06a
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-05-17 10:24:48 -07:00
Dariusz Stojaczyk
7463bf6d2a vhost: add negotiated_features field in spdk_vhost_dev
Added new struct field and an rte_vhost_get_negotiated_features() call.

Change-Id: I5937026720b5341c7377c27fee682996b51e836e
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-05-17 10:24:48 -07:00
Jim Harris
737b1b571b build: add dpdk as a submodule
This will allow a git clone to default to an SPDK-specific
version of the DPDK repository.  Users can still override
to use a separate DPDK repository/installation with the
--with-dpdk configure script options.

While here, remove gzip option for the git-archive operations
in autopackage.sh.  We need to add a git-archive for the DPDK
submodule if we are using it, and compressing at -9 adds a lot
of unnecessary time.  Since we are not archiving these packages,
there is no need to compress them.  Also explicitly disable
coverage and ubsan for the autopackage build, since this build
is only to test compilation and is not actually used for any
test execution.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3cf8a2ed984003a175cdece6542636ede8cb2479
2017-05-17 09:49:27 -07:00
Jim Harris
8de75f8107 nvme/perf: add software-based latency tracking
The latency tracking is done with ranges of bucket arrays.
The bucket for any given I/O is determined solely by TSC
deltas - any translation to microseconds is only done after
the test is finished and statistics are printed.

Each range has a number of buckets determined by a
NUM_BUCKETS_PER_RANGE value which is currently set to 128.
The buckets in ranges 0 and 1 each map to one specific TSC
delta.  The buckets in subsequent ranges each map to twice
as many TSC deltas as buckets in the previous range:

Range 0:  1 TSC each - 128 buckets cover deltas    0 to  127
Range 1:  1 TSC each - 128 buckets cover deltas  128 to  255
Range 2:  2 TSC each - 128 buckets cover deltas  256 to  511
Range 3:  4 TSC each - 128 buckets cover deltas  512 to 1023
Range 4:  8 TSC each - 128 buckets cover deltas 1024 to 2047
Range 5: 16 TSC each - 128 buckets cover deltas 2048 to 4095
etc.

While here, change some variable names and usage
messages to differentiate between the existing latency
tracking via vendor-specific NVMe log pages on Intel
NVMe SSDs, and the newly added latency tracking done
in software.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I299f1c1f6dbfa7ea0e73085f7a685e71fc687a2b
2017-05-17 09:49:27 -07:00
Daniel Verkamp
a591161cb2 bdev: make struct spdk_bdev contents private
Change-Id: If203e82f8cd10d5998a565ad490ef11e2916687f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
eaf9019738 nvmf: use bdev name getter
Change-Id: Id0094871bb18f90908f3a9d96f47ddd264cea5bd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
9d1e05d2f2 bdev: add getter for write cache enable status
Change-Id: I93656deda8956c77e9565ee1f271795642e62ac9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
0d56499d3a scsi_bdev: use max unmap descriptor getter
Change-Id: Id67b158c36b2b879849b18909bc9bb9a5273f8c7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
76e67baf28 blob_bdev: use bdev block size getter
Change-Id: Id22f904fdb71798ef2369415d38f468b06d7ac07
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
ed362f1f35 test/bdevio: remove gencnt manipulation
The bdev generation count is an internal implementation detail;
applications should not be reading or writing it.

Change-Id: Ic4455b7b72dc80babbc410420c89f52ef5d77e6d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
5bb041adcf bdev: rearrange spdk_bdev_io
Move some of the fields in spdk_bdev_io (the layout of which is no
longer public API) to put related fields together and improve struct
packing.

Change-Id: I0ca5d9441b00c65703c35391d125059eaa4bcee3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Jing Xia
928df4a448 test/blobfs: Rename blobfs test file name
Change-Id: I584c667aff5a0cdfee26b958a2b58c6ed41d45f7
Signed-off-by: Jing Xia <jingx.y.xia@intel.com>
2017-05-17 08:21:59 -07:00
Daniel Verkamp
133dc1e6ae iscsi: clean up iSCSI/SCSI task conversions
Add a helper function to find the iSCSI task given its nested SCSI task
structure, and use it to remove all casting between spdk_scsi_task and
spdk_iscsi_task.

Change-Id: Idc7c6d0a3b4d53041916d25a1bdecedfb56b94f9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-16 16:45:22 -07:00
GangCao
b347d551e8 nvme: detach the pci device with calling process's own devhandle
Change-Id: I2693b4bd29e0500379d5e399723aec7e44bffca3
Signed-off-by: GangCao <gang.cao@intel.com>
2017-05-16 10:35:29 -07:00
Ziye Yang
644678258f blobfs: move the location of next buffer check.
This patch address the issue:
https://github.com/spdk/spdk/issues/151.

For cache_append_no_cache in cache_ut testcase,
there is resource contention for buffer among two
threads in the following two functions.
Thread 0: cache_free_buffers
Thread1: __file_flush_done

When the thread1 execuctes __file_flush_done,
it calls the call back: __sem_post defined in
following statement in spdk_file_sync

_file_sync(file, channel, __sem_post, &channel->sem);

Thus Thread 0 will execute next function
cache_buffers, and it frees the buffer.

Then Thread 1 continues executing the remaining statements
in __file_flush_done with the assert function, and touches
the space already freed.

So it will be safe to move ahead the next buffer check.

Change-Id: Ic007b3481f4e3a17d47eeca5c9c802001949a5ab
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-05-16 10:51:26 +08:00
Jim Harris
4eafea0360 bdev: add a null bdev module
Also change the discovery/nvmf.sh test to use it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I56bce9a84bd46f13b6d4f34da81abf23413f2598
2017-05-15 14:00:37 -07:00
Daniel Verkamp
7455910d1b scsi_bdev: simplify scsi_readwrite helper function
Factor out the common data direction Check Condition logic from read
and write.

Change-Id: Ia2fbd5f69eefd826ebb5ab2a8e99aeb380f1b7ea
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 13:35:18 -07:00
Daniel Verkamp
8e379851ab scsi_bdev: clarify block count variable names
What was previously called "maxlba" was actually the block count
of the device, which is the maximum LBA plus one.

Change-Id: I5e141a3eb0c79b2aa70006739c5586cb50744d3d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 13:35:18 -07:00
Daniel Verkamp
658f816344 test/scsi_bdev: add read LBA range tests
Change-Id: If0cc8f970c20c4a398882183de9e7b066753f030
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 13:35:18 -07:00
Daniel Verkamp
19a9871495 scsi: remove spdk_scsi_task::owner_task_ctr
The SCSI library already provides a callback when the task is released
(free_fn), so the user can update their own task counter.

Change-Id: I7fb13f6fff66dbba2315fd03fb06e49f793be123
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 10:12:42 -07:00
Daniel Verkamp
7145cf62c6 scsi: set free_fn in spdk_scsi_task_construct()
The task free callback function is required, so make its assignment part
of task construction.

Change-Id: I2f5fdf73b064653ee85b4e7961cb1653a0a4107d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 10:12:42 -07:00
Daniel Verkamp
1aea6c5b6f bdev: add getter for maximum unmap descriptors
Change-Id: Ifbdd29e2081600bf0d860582d80696546107cf1b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 10:11:23 -07:00
wenzhong wu
8651c2eb47 nvmf: call function spdk_nvmf_valid_nqn to check the host_nqn.
Change-Id: I035fe2538b46d0a19eb00fd829fae9e735c1ded9
Signed-off-by: wenzhong wu <wenzhongx.wu@intel.com>
2017-05-16 03:19:54 +08:00
Daniel Verkamp
326786a943 bdev: add getters for block size and block count
Change-Id: I6fad28da43c163ea4e2c4a04ced356b67d63652f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-12 10:34:58 -07:00
Daniel Verkamp
1bcf22cde1 bdev: add getters for bdev name and product name
Change-Id: I6a75fada94fa845ecedd4cd8afc78f4259df14fe
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-12 09:36:30 -07:00
Dariusz Stojaczyk
6973898164 vhost: wrap vhost_user_read_cb with vhost_user mutex
This fixes spontaneous vhost hangs on SIGINT shutdown.
Apperently during vhost_destroy_device(conn->vid) from
line #284 another QEMU message might arrive, causing
vsocket->conn_mutex deadlock. (line #286)

Change-Id: I4f1c31a52facffd1eb1e1192591095f00da55031
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-05-12 08:05:37 -07:00
Daniel Verkamp
f4cd3c08dd autotest.sh: run NVMe hotplug test on every build
Change-Id: Ic15046490d1da625bb9a17d6ef3e7b27bbfbfe00
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-12 08:04:53 -07:00
Pawel Wodkowski
66894635fa vhost: obey VRING_AVAIL_F_NO_INTERRUPT
If VRING_AVAIL_F_NO_INTERRUPT flag for queue is negotiated, guest can
set this flag in avail ring to suppress interrupts from host.

Change-Id: I2043067de33da009a974be5915ae81b1343fc424
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2017-05-12 07:53:59 -07:00
Paul Luse
bf31637789 nvme: add timeout value to multi-process driver init
Unit tests will be added as part of a separate patch updating all UT for
nvme.c.  Global used for timeout value so it can be easily overwritten
by the upcoming unit tests for this function.

Change-Id: I7fc15aab91601ac57c94cae266b212c0998d2495
Signed-off-by: paul luse <paul.e.luse@intel.com>
2017-05-12 07:53:48 -07:00
Changpeng Liu
c7f6f553b9 env/dpdk: use the new *rte_pci* API for DPDK 17.05
Change-Id: I1091a9556bbd42349dd8d26f238a54a5b55610a1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2017-05-12 07:53:42 -07:00
Jim Harris
54a5aba7ee detect_cc.sh: further fix CC_TYPE generation
FreeBSD clang prepends "FreeBSD" before clang in the
version string, so further fixes are needed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7bfc37322483b97443985f1103ff753d1b18a87a
2017-05-11 16:45:24 -07:00
Daniel Verkamp
905551da05 nvme/fio_plugin: remove dead store
Change-Id: I69d614434e6ce1dd16eb0009d7ededc7be807d09
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-11 16:03:51 -07:00
Daniel Verkamp
93f7d910c6 nvme/fio_plugin: make file-level variables static
These variables are not needed outside of this file.

Change-Id: I577db5140fb4156100e72e6219c8121304a9b137
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-11 16:03:51 -07:00
Daniel Verkamp
dba5490000 nvme/fio_plugin: make setup callback thread safe
Change-Id: I40f858a204bf0ca5e594fc8efcb1a31849e6709b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-11 16:03:51 -07:00
Daniel Verkamp
4325814761 scripts/detect_cc.sh: make C vs C++ warning non-fatal
Even if we detect that $CC and $CXX are different types, we should
continue so that mk/cc.mk is still generated.

Change-Id: I551e2d9149ea982d3c9973dfc5f2557f12fbc337
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-11 13:46:46 -07:00
Changpeng Liu
6c0d4c552b env/dpdk: return value changed of API rte_ring_mp_enqueue_bulk for DPDK 17.05
Change-Id: I44f1c1fc75fd343d30c602ca77940ebde3ed184a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2017-05-11 12:45:31 -07:00
Ben Walker
58afa58a00 bdev: Simplify spdk_bdev_get_io_buf
This was implemented as two functions, but it
is much simpler as one. Also, the public function
was way at the bottom of the file instead of near
spdk_bdev_put_io_buf.

Change-Id: I3a90688910b0542cc77b6333bab15132cf514eeb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-11 10:48:58 -07:00
Ben Walker
812ffaf1a3 bdev: Simplify spdk_bdev_finish
This was implemented as 3 separate functions but
it is simpler as 1.

Also, this wasn't previously freeing the buffer pools.

Change-Id: Ic1b2b3a0596e745a223099cb2a79bea6ef5c69cc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-11 10:48:58 -07:00
Ben Walker
f2132bfd8a bdev: Simplify spdk_bdev_initialize
This was broken into three functions, but it is
a lot simpler as one.

Change-Id: If58ad50fe7d4f65c598b62f24e9e1ce7a64fdd8e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-11 10:48:58 -07:00
Ben Walker
ba17fd1bf2 bdev: Move all globals into a single struct
This is better organizationally, but also will serve as
an io_device in the future.

Change-Id: I6d65cf39df59e874d13f5fccc5a489720e86c48f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-11 10:48:58 -07:00
Jim Harris
3510268bf7 detect_cc.sh: fix CC_TYPE generation
Some compilers will emit version info with a --with-pkgversion
parameter.  So grep for "version" with -w.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7bfc37322483b97443985f1103ff753d1b18a879
2017-05-11 10:22:08 -07:00
Daniel Verkamp
8d24e2da5b test/nvme: support systems with non-0 PCI domains
Fix up a second copy of linux_iter_pci in the same manner as commit
6562e95092 (scripts/setup.sh: support
systems where more than one domain is used).

Change-Id: I3d9b842891d70c2960de8287e3b11c1a11b02d1f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-11 10:12:49 -07:00
Daniel Verkamp
94a2bec4a8 nvme/fio_plugin: remove unused MAX_LCORE_COUNT
Change-Id: Ie9015d88a8919885c620b51fac61ea49e120ac68
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-11 09:32:39 -07:00
Karol Latecki
ac9d99071e vhost/test: SIGKILL vhost after unsuccessful SIGINT
Added 1 minute timeout for vhost to exit after receiving
INT signal. After timeout send KILL signal to vhost app
and exit with error.

Change-Id: Ib1659660667991b21103b3401bbd780290521433
Signed-off-by: Karol Latecki <karolx.latecki@intel.com>
2017-05-11 01:27:31 -07:00
Xiaofan Yang
d8369fee70 iscsi/conn: Fix the bug when iterates the read command subtasks.
Change-Id: Ib11e804977b6057bcb9cc6d3a2d21ec43b45a99a
Signed-off-by: Xiaofan Yang <xiaofanx.yang@intel.com>
2017-05-11 08:09:23 +08:00
Daniel Verkamp
becdc9a73d scripts/detect_cc.sh: add CC_TYPE to mk/cc.mk
Change-Id: I5d627629d545c4d9fe366f638bbeb812bc3add95
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-10 13:46:08 -07:00