Spdk/test
paul luse a0e22e1303 test: add 'match' util and use it with blobstore
Will follow this up with a doc change but want to make sure we're
all good with it first. This is meant to not only beef up
blobstore testing but provide the 'match' utilitiy for all test
cases where we are currently calling an executable and either
counting only on a return code to determine success or failure
or worse yet we're just running it and if it doesn't explode we
assume its a pass.

The 'match' util was borrowed from the PMDK folks after first
adding the "ignore" feature upstream to make it easier to use
in SPDK.  It works like this:

When the developer checks in a test they create and check in
the output of the test with two different file extensions:

.ignore: should include a string per line for output lines
that we want to totally ignore typically because they're
platform specific so the output could be different from
machine to machine.  In this case I'm ignoring all output
lines with 'DPDK' or 'EAL' or '...' in them. The first
few are obvious, the last is because the test tool will
print a varrying number of these as progress indicators.

.match: this is a copy of the output that the developer
'fixes' up by replacing platform specific output strings
with replacable tokens as described in the 'match' help.
This is where you'd want to match an entire line minus
something like a CPU count or free block count or
something. The 'ignore' feature was added simply so we
wouldn't have to edit every single line of an output
file that had DPDK or EAL in it.

Then you modify the test script to save the output and
smply run the match util providing the name of the
match file and if it fails to match the actual output
with the saved output that's been token'ized the script
will error.

The obvious advantage here is that now we can confirm all
of the output from a test executable is as we expect.

Change-Id: I701d36ee83d37b6193e16ed3171e114f814e5eb3
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/397027
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-02-06 18:05:25 -05:00
..
app test: add 'match' util and use it with blobstore 2018-02-06 18:05:25 -05:00
blobfs/rocksdb test/rocksdb: reduce test time for per-patch tests 2017-12-15 15:48:06 -05:00
blobstore test: add 'match' util and use it with blobstore 2018-02-06 18:05:25 -05:00
config vm_setup: add required packages 2018-01-26 12:57:40 -05:00
cpp_headers build: include spdk.app.mk in all app Makefiles 2017-11-15 17:57:07 -05:00
iscsi_tgt test/iscsi: check and avoid setting DataDigest 2018-01-24 16:52:30 -05:00
lib test/bdevperf: fix memory leak if opening a descriptor fails 2018-02-06 10:22:48 -05:00
lvol test/lvol: Fix test cases for lvol store cluster size 2018-02-06 10:53:46 -05:00
nvmf nvmf: Add rpc to add listeners to subsystems 2018-02-06 10:29:42 -05:00
pmem bdev/pmem: require name in construct_pmem_bdev RPC 2017-12-04 18:51:14 -05:00
unit bdev: add a common channel resource destroy function 2018-02-06 11:06:28 -05:00
vhost test/vhost: check vhost return code in spdk_vhost_kill 2018-02-06 14:50:39 -05:00
Makefile test: begin moving unit tests into test/unit 2017-06-16 16:43:48 -04:00
spdk_cunit.h include: Move the remainder of the code base to stdinc.h 2017-05-08 13:20:36 -07:00