per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the 4 digit year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.
Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.
Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".
Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>
While here, replace use of "slave workers" in some
comments with "secondary workers".
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2169c108da18d449a66a29daa77a3f9c3145d4b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5352
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This change will make sure that ioat_perf application
exits with error code 1 whenever there was
no test performed.
Mainly by checking number of ioat channels available.
Other changes made to align with returning 1 on error.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id94fa947fd097d5085285d3348aeb41ff280e731
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1672
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This avoids conflict with the nvme perf tool. PGO
gets confused during building - we may have data for
nvme/perf which it then tries to use when building
ioat/perf. Renaming the ioat perf tool fixes that
problem.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib1084d56d671e44027ea05f453075a723f067580
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455320
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Currently the task and buffer pools are hardcoded to
512 each. This effectively limits the queue depth
per channel to 256 since we need both source and
destination buffers from the buffer pool. So make
the pool sizes dynamic based on the user's queue depth
input.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I52f04308c0329fa99277746a0768eb1214e37a8e
Reviewed-on: https://review.gerrithub.io/c/445357
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Use the new spdk_ioat_flush interface to batch
doorbell writes - this significantly improves
descriptor throughput. For now, just set the
threshold for writing the descriptor to half of
the queue depth. We can always modify this later
to allow very specific thresholds, but for now
this simple change is sufficient.
Increases 512B descriptor throughput at QD=256 from
3.7M/s to 14.0M/s on my Skylake Xeon server.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4504579e23cee5b6a1044849c49d33d58bdb51a9
Reviewed-on: https://review.gerrithub.io/c/445355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I3b3ca9ce1db609bda59e991f84f511747f49eb87
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433825
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
'-s' option is usually associated with the memory size,
but our ioat/perf and nvme/perf use it for io size.
To keep those consistent:
'-s' option (io size) was be changed to '-o'
'-d' option (mem size) was be changed to '-s'
Change-Id: I97cef4c0a4e6e902277dd50dfb43bde420a6427e
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Automatically detect more whitespace errors.
All existing cases are fixed; only whitespace change (verify with
diff -w) except for one comment style fixup in include/spdk/nvme.h.
Change-Id: If750e54b9c8e3421ea6feda5f20184a31431631e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402360
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Modifies spdk_env_init() and spdk_mem_map_init() such that
they return on failure instead of terminating with exit()
or abort().
Change-Id: I054c1d9b2e46516ff53d845328ab9547f54bdbc4
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/393987
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Use the env.h abstractions instead.
Change-Id: I273ee0182b9da51f762cdd6af56056d08dcca9a7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/389401
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is just a convenience and replaces the common practice
of passing -1.
Change-Id: Id96734307ebf52ef0ee7dba0e7ac89602b2b5b1a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/374520
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
DPDK's use of getopt() needs special handling of the optind global
variable since we are passing it a separate array of arguments (not the
typical argv and argc). Set optind to 1 internally to env_dpdk so that
the apps don't need to know about it, and restore optind in case the
calling app is also using getopt().
Change-Id: Icbf07002c99fa9f94c866e8eff707124b0ef679b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/365062
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The snprintf() calls could overflow the buffer if i is very large.
This is not possible in reality since there are only a few I/OAT
channels on real systems, but it fixes a GCC 7 warning.
Change-Id: Ia202872135cc538120e17be2c76e16b687f3ee37
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/364302
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I13f4be0d4b41e1f8b6fe5dd10e988e111f596999
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/363606
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: John Meneghini <johnm@netapp.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
- rename spdk_malloc_socket to spdk_dma_malloc_socket
- rename spdk_malloc to spdk_dma_malloc
- rename spdk_zmalloc to spdk_dma_zmalloc
- rename spdk_realloc to spdk_dma_realloc
- rename spdk_free to spdk_dma_free
Change-Id: I52a11b7a4243281f9c56f503e826fd7c4a1fd883
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/362604
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Several examples have a function to associate workers
with threads. Simplify that algorithm.
This seems to just shift some of the complexity
from register_workers down to main, but in the long
run the DPDK threading will get abstracted into
env as well and greatly simplify that part.
Change-Id: Ic106dde58fa5351a1ce0a058161b08062e121d3b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
If no channels are available, don't try to run the test.
Change-Id: I4c40635a3da598064da7c94b3c7960a83ba25f8f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Previously each core can only leverage one ioat channel.
With this patch, we can support the following features:
(1) Users can input the number of ioat channel to be tested.
(2) If the number of CPU cores is c, the total ioat channels
are n, each cpu core can use n/c channels.
Change-Id: I6b2179d3d633cb0999a3f4c5f40d4605c8cebc45
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
It always returns NULL in the current DPDK env implementation and was
not used outside of a few ioat examples where it is not particularly
informational.
Change-Id: I14b237c33bc25ddebc6b36bfbd6a4edf6762e3ca
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Remove #includes for all DPDK headers that weren't
necessary.
Change-Id: Ib02522e0f04e64a1c98afceb7508cc0e8d931a9d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This converts some, but not all, usage of rte_mempool
to spdk_mempool. The remaining rte_mempools use features
we elected not to expose through spdk_mempool such as
constructors, so that will need to be revisited.
Change-Id: I6528809a864ab466b8d19431789bf0f976b648b6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Use the env library to perform all memory allocations
that previously called DPDK directly.
Change-Id: I6d33e85bde99796e0c85277d6d4880521c34f10d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This allows users to swap their PCI library from
libpciaccess/dpdk to another mechanism using the standard
method for swapping out the env library.
Change-Id: Ib2248f8b43754a540de2ec01897e571f0302b667
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.
Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.
Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Add a parameter to each I/OAT library function that requires a channel
instead of implicitly using the thread-local channel registration model.
I/OAT channels are already reported by the spdk_ioat_probe() attach
callback, so no infrastructure for channel allocation is necessary.
Change-Id: I8731126fcaea9fe2bafc41a3f75c969a100ef8f0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Rename all functions with a spdk_ prefix, and provide enough of an API
to avoid apps needing to #include <pciaccess.h>.
The opaque type used in the public API for a PCI device is now
struct spdk_pci_device *.
Change-Id: I1e7a09bbc5328c624bec8cf5c8a69ab0ea8e8254
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Similar to the NVMe API change, this allows better abstraction of the
PCI subsystem.
Change-Id: I2b84d9c3c498a08d4451b4ff27d0865f0456c210
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Previously, ioat_init() in the example programs was returning 0 even if
initialization failed.
Change-Id: I96b2ec5646f7051ab881611acff424fb8547d5eb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Also add a space between Copyright and (c).
The copyright year can be determined using git metadata.
Also remove the duplicated "All rights reserved." - every instance of
this line already has a corresponding "All rights reserved" immediately
below it, except for examples/ioat/kperf/kmod/dma_perf.c, where I have
added it manually.
Performed using this command:
git ls-files | xargs sed -i -e 's/Copyright(c) \(.*\) Intel Corporation. All rights reserved./Copyright (c) Intel Corporation./'
Change-Id: I3779f404966800709024eb1eb66a50068af2716c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This more accurately represents what function it performs.
Also remove pci_device_has_uio_driver() from the public API. Callers
should use pci_device_has_non_uio_driver() instead.
Change-Id: I9623fe1345b43e981d5823804e33d01ac0d3bb1c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Previously, the cleanup functions were only being called if there was an
error during initialization.
Change-Id: I1606cfa9a9c3732d670131f78249d34a5db47403
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The ioat driver supports DMA engine copy offload hardware available on
Intel Xeon platforms.
Change-Id: Ida0b17b25816576948ddb1b0443587e0f09574d4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
