This is meant to improve performance of the heaviest checks, like
the shellcheck one, where instead of running against the contents of
the entire repo simply go through files that are meant to be
committed.
This mode has to be requested through the environment via the
CHECK_FORMAT_ONLY_DIFF.
For now, get_diffed_dups() is hooked into least performant
checks (permissions, shfmt, shellcheck).
Also, the git grep command in the main check_permissions() is dropped
as the idea of piping its output to git ls-files was faulty - the git
ls-files does not read its arguments from the stdin.
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I098dcbbe18c08c08a8216857c7cf2c80c4021d31
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16049
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I0d38256afb918f1273def36b3d4915d5a1911df9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15422
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.
git log --follow -C90% --format=%ad --date default <file> | tail -1
and then pull just the year from the result.
Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)
For intel copyrights added, --follow and -C95% were used.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Look for all declarations instead of only the ones introduced in the
tested patch. It makes this test pass if a patch adds definition for a
function that already has a declaration in the header file (e.g.
the implementation depends on some #ifdef).
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I735cf10d4524318009f8c4282a86e4137bf8c769
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
bdevperf application is utilized in multiple test scenarios
and most prominently in SPDK performance reports.
Similar to perf and fio_plugins, it is used to measure performance.
It is intended to represent the expected behavior of users
application that use SPDK bdev layer.
Applications under --enable-tests are intended for specific test
scenarios and user interaction is usually via a test scripts.
Meanwhile --enable-examples are used more broadly to teach and
show usage of SPDK libraries.
As an added benefit with this patch, its possible to compile
bdevperf without need to satisfy additional requirements to
compile all the test applications.
Change-Id: I9aaec1f9d729a1cdee89b5fdc365d61c19b03f82
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14558
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Moving forward, we want to still be able to run against
<= 22.07 versions of DPDK, which exposed the necessary
data structures in public header files. But since we
will be building against newer versions of DPDK which
don't expose them publicly, we need a copy of the 22.07
header files in our tree.
Exclude these header files from astyle and POSIX include
file checks in check_format.sh
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icd8a067af41a2ba031ce8f875a8a2b63f722ab69
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14683
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
We don't track any files in test/cpp_headers, so there
is no need to do a grep -v to exclude files that
contain "cpp_headers". This was a remnant from before
we started using git ls-files to determine which files
to run through astyle.
Fixes: fb87f80c ("scripts/check_format.sh: only check tracked files")
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If08f0bd5c96990dc5e4ca4641c53f70a82590470
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14689
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This was a remnant from ages ago when we had rte_vhost
DPDK code copied into our repo. We actually have a file
named rte_vhost_user.c which is not DPDK code that was
getting excluded from astyle checking.
So this also includes the astyle violations that had
crept into this file. In a couple of places, change
the enum return type to int, this reduces astyle
confusion on function and if brace style.
Same applies to POSIX include checking - we don't need
to exclude rte_vhost_user.c from this either.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If3a25011ad54c694c15a91f7be66d862c765c5db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14688
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
A new RPC perform_tests has been added.
This request will help us avoid reading hotplug generated
logs, and instead we can stop hotplug right before
it begins generating IO, and resume it when the test
scripts are ready.
Additionally a new command line option has been added
to the hotplug application "--wait-for-rpc", which
indicates that hotplug should wait for
perform_tests RPC before starting its IO.
Change-Id: I71ca148201854ac155cc2a61171a4fb5fc427a19
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13962
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
glibc 2.36 added arc4random(), which breaks
the SPDK iSCSI build since it always implements its
own arc4random() implementation on non-FreeBSD OS
(meaning always on Linux).
So instead add a CONFIG_HAVE_ARC4RANDOM and remove
the explicit FreeBSD dependency - this will work on
FreeBSD as well as Linux with >= glibc 2.36.
Also fix check_format.sh, so that it does not
enforce spdk/stdinc.h checks on code snippets in
the configure file.
Fixes issue #2637.
Reported-by: Karl Bonde Torp <k.torp@samsung.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab9da8ae30d62a56869530846372ffddf7138eed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14028
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
This was temporarily disabled to work around a big style fix patch. It
can be enabled again now.
Change-Id: I9009d83798551545307cf5881bc1bbb87453e611
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13466
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enforcing this. Unfortunately, it seems to have two bugs:
1) It doesn't work correctly at all on C++ files.
2) It often fails on functions that return enums, or long type names
Deal with 1) by adjusting the check_format.sh script to only tell astyle
to fix return type line breaks for C files and not C++. Deal with 2) by
adding a few typedefs to work around the problem.
Change-Id: Idf28281466cab8411ce252d5f02ab384166790c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
This check isn't smart enough to handle the next big formatting patch,
so disable it and then re-enable later.
Change-Id: I2d73598d6b7c43f9356562161ed116bc01361166
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
These are going to need different options in an upcoming patch.
Change-Id: I5d43f21887abefbae207b5cff23fafcfac4a33d7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13464
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Our CI explicitly uses v3.1.0 shfmt. Other versions,
including newer versions, may suggest a lot of script
changes that are different than what v3.1.0 reports.
So change check_format.sh to *only* run shfmt
checking if the version is exactly v3.1.0.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I788961faaae7992cf1369abf1e54f4075bd2521c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12309
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If a function prefixed with spdk_ was moved between
files, the check_format.sh naming convention checks
would fail. This is because it thinks the function
was added, but doesn't see it getting added to the
header file, since the header file wasn't touched
by the commit.
So resolve this by doing the defined/removed checks
on a per-library basis, rather than per-file. The
checks already handled the case where functions
were moved within a file, and that will all work
the same now that we check on a per-lib basis.
Fixes issue #2307.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If85a1e9c3cd349b701a10531726e814b60fba26d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10967
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
check_format.sh requires the following, none of which
are supported by default on MacOS and require alternative
homebrew installation for GNU variants:
* mapfile command not supported by bash
* -f option for readlink
* -P option for grep
Note that SPDK is not supported on MacOS, the changes
here are added only for developer convenience.
Fixes issue #2255.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic3d4ed49d9bfb4be50a8dd090a34090037f592c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The check needs to fail if documentation is missing
for any RPC. But the way the check was written, the
rc would get set to 0 everytime it found docs for
an RPC. This means it would only return failure if
the last RPC that was checked did not have
documentation.
While here, add a message telling the user that
JSON-RPC docs are being checked, to be consistent
with other checks in this script.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib2ff0f0432ce3e2853526223c0ad00176d84a78c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9391
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
SPDK has been inconsistent in the argument names,
most use dashes (--max-subsystems, --io-unit-size),
but in some cases we had underscores (--tgt_name,
--include_aliases).
So let's do 3 things here:
1) convert all argument names to use dashes only
2) if user passes and argument with an underscore,
convert it to a dash implicitly - this ensures
backward compatability and is a bit less pedantic
3) enforce use of dashes in check_format.sh
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99289d3d3b03ce40cf192ad4fddec7b5c6bc8388
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9390
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
MD025 - Multiple top level headers in the same document
Fixed all errors
Update check_format.sh to fit new header style in jsonrpc.md
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib5f832c549880771c99c15b89affe1e82acd3fa4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9045
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
-g option using only files known to git, that allow us
to avoid errors from submodules
Also add check if mdl is installed, and gracefull info if not.
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib6e1920774ffca81e62d9abebc8d8b4548feb519
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9086
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Use extended regexp to resolve any potential ambiguity with matching
on '('. This should fix the following failure as seen on the CI:
fatal: command line, '^SPDK_RPC_REGISTER\(': Unmatched ( or \(
Also, since errexit doesn't see failures inside the process
substitution make sure we return from the function with a proper $rc
in case git fails early on.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia03095e9cc8cf11602dafb5bef28265abb485704
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8577
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
These are currently prone to false-positives especially in terms of
scoping. Consider the following:
foo() {
local bar=(42)
echo "${bar[*]}"
}
bar=43
foo
echo "$bar"
Some versions of shellcheck, including the latest, 0.7.2, complain
about $bar being reused as a plain string here. This is incorrect
since foo() holds its own copy of bar[@] hence the assignment which
takes place outside of it doesn't affect its content.
SC2178 can be mitigated be reversing the order of declaration:
bar=43
foo() { ... }
...
but the SC2128 still remains.
Currently, in our code majority of these warnings are coming from
false-positives due to initial source'ing which most of our test
scripts do (e.g. they fetch a function where local bar=() is used
and in the test itself $bar happens to be assigned a plain string.
This is still valid code).
To mitigate, disable these directives untill shellcheck is capable
of properly interpreting scoping when checking them.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ifbde973eae6e261d79e1c340eb28644bce5f4e45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8503
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a first commit in the series addressing potential Bash issues
as discovered by the latest shellcheck release (0.7.2, shipped with
the very latest fedora33 and fedora34). The goal is to either fix,
locally or globally disable given directive(s).
SC2155: Declare and assign separately to avoid masking return values
Simplify the setting of the variable pointing at the root of the repo.
Also, keep it consistent with the rest of the scripts and declare it as
$rootdir.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I63e0b1a85ce16f7983e9ba6dd985046e8a39a650
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
On my system, the git grep for ^SPDK_RPC_REGISTER(
fails because the ( is unmatched. It seems we should
escape it, although clearly our CI systems don't seem
to require it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6412e27e79488616743f76b2d9eac8f62b996078
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8393
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
All JSON-RPCs must be documented in jsonrpc.md
Change-Id: Iec9119a01433fd0cbe4df4477ce94cc2feeb0d54
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4074
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Community-CI: Mellanox Build Bot
-x is supported since 0.4.0 release, make a note of that.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Icc64132536c18108029f6c5fa2cbdd5b27018d8b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7417
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Currently, reference repos which are used to clone the SPDK repo in
the CI pool's systems don't provide master head refs, hence there's
no easy way to diff against it.
Instead, simply run shfmt against all the bash files in the repo.
Impact on the performance should be minimal.
Also, cleanup some related code which is not used anymore.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie0d4e35b8ad214ceed1b4ea29b01a7423b8ff73b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Checking only the .sh suffix was omitting ./configure and potentially
other bash scripts which don't necessarily have to include said suffix
in their name.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6196559e8875de46f80d9c8426577a1a79b5996d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Original code removed $symbol prefix from every entry in
$defined_symbols. This was incorrect shown by example
patch next in series.
Original function spdk_json_decode_object() was moved
and spdk_json_decode_object_relaxed() was added.
This resulted in $defined_symbols containing entry "_relaxed".
Now each entry in $defined_symbols is checked separatly to
match exactly to the removed $symbol.
Reported-by: Jacek Kalwas <jacek.kalwas@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1d9931d2e93dc85465ce47a838a176c6ab5f1587
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4357
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
shellcheck.log
shellcheck does not seem to reliably return error codes, at least
on the 0.7.0 version packaged on Fedora. Instead, look at
whether shellcheck.log contains any output.
Change-Id: Iec3c4f416844eed6a6c75b9204e24c2bc7243f7a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Even on version 0.7.0, the diff mode throws up a bunch of errors
that don't appear in the tty mode. We'll have to deal with those
at a later date.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ic7001b9cf793e696bbf4bab0b0630e8d705a6905
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4071
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
I had previously implemented a portion of the naming convention
check that looked for symbols that had been moved by performing
two checks:
git diff commit1 commit2 -- libname | grep for added function
git diff commit2 commit1 --libname | grep for added functions
and then subtracting the values returned by the second check from
the values returned by the first check. That works as long as the two
diffs are reciprocal (i.e. the first diff is a mirror image of the
second). However, this has proven to not be the case.
This change fixes that check by performing the smae diff twice and
grepping for removed functions the same time.
Change-Id: I09c81921d68436baeee706f2d9a6d30db1d23976
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3229
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Make sure that git diff ignores all submodules while looking for
changes made by astyle. This should prevent the following issues
from occurring:
object directory /var/ci_repos/dpdk-submodule.git/objects does not exist; check .git/objects/info/alternates
object directory /var/ci_repos/intel-ipsec-mb.git/objects does not exist; check .git/objects/info/alternates
error: bad tree object HEAD
fatal: 'git status --porcelain=2' failed in submodule intel-ipsec-mb
Change-Id: Ie9a39e324674337fdbd6bf627da6e80775ceaeec
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3223
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In a few tools or languages, slash(/) is maybe a delimiter by default in regex.
An unescaped delimiter must be escaped with a backslash (\).
Otherwise the pattern of regular expression will be error
when delimiter is slash(/) in a few special environment, i.e. in MinGW32 git bash.
Signed-off-by: Dayu Liu <liu.dayu@zte.com.cn>
Change-Id: I73b556416b98e135eb240078dcc68f80e08c5e74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2488
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There was an edge case in this check where a public symbol
definition was just moved and that movement triggered this
check. Going forward, check the diff for symbols that were
both added and removed in the same patch. This indicates a
symbol that was just moved instead of newly added.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I71bbcd6d6b3e0a2133e77c29f4ec7a4f2b09e3c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
It turns out we have a function to compare version numbers.
It additionally handles cases like "3.0.10 > 3.0.9". The
previous code returned false, the new one returns true.
version_lt() actually tests for less or equal, so compare
against 3.1.0 - the real minimum shfmt version we need.
Change-Id: Ibd18c4505e5b81c0073a907f23c3fc034cfbafbe
Suggested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2574
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Previously when doing the naming conventions check, we were
checking against master which was causing some weird issues on
patches which weren't rebased. The much simpler check is to just
compare each patch against the previous one.
This method still works to prevent any bad changes from getting
merged, but handles naming issues on a patch by patch basis.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic031ce07ca1c67819b792dc292dcf6c50df5b1a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
If $output_dir is set, putting the shfmt diff there will make it
easier to extract it and/or look it up from the side of the per
patch build.
Change-Id: Iefb601a29f165e59b7670a8878e95bb6b16c9650
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2277
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This test prevents anyone from adding a function that begins with
spdk to the lib or module directories without also adding that file
to the corresponding map file and a header in spdk/include or
spdk_internal/include.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ife7db13d4bb8fec7570c8492ac3326222a850c36
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2423
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
All of the other checks simply print OK if the test passes
this check prints out a lot of text (especially when run at the
tip of long patch series) that can make it difficult to tell if
anything is actually wrong.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ie681d200aa13f208e4afa1ab419cbad63ef6606c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2403
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
A newer version of shfmt was released, but the script still
looked explicitly for the older one.
Make it execute the first binary '^shfmt.*' in PATH whose --version
returns at least v3.1.0.
Change-Id: If621abfa4d995a9dc985e6bab8c33aa36c327fae
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2428
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add new dev tool for enforcing proper formatting of the Bash code
across the entire repo. This is done in order of defining a common
set of good practices to follow when writing .sh|Bash code.
As powerful as shfmt may be, it allows only for some specific rules
to be enforced, hence it still needs to work side by side with
shellcheck syntax-wise. If it comes to style, following rules are
being enforced:
* indent_style = tab - Lines must be indented with tabs. The exception
from this rule is the use of heredocs with
<<BASH redirect operator. Spaces can be used to
format the line only if it's already preceded
with a tab.
* binary_next_line = true - Lines can start with logical operators. E.g:
if [[ -v foo ]] \
&& [[ -v bar ]]; then
...
fi
* switch_case_indent = true - case|esac patterns are indented with tabs.
* space_redirects = true - redirect operators are followed with a space.
E.g: > foo over >foo.
In addition, shfmt will enforce its own Bash-style for different parts
of the code as well. Examples and more details can be found here:
https://github.com/mvdan/sh
Change-Id: I6e5c8d79e6dba9c6471010f3d0f563dd34e62fd6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1418
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
