ada6c8468c
The target changed how we deal with custom AERs so we can't post them and expect to get an immediate return anymore. The fuzzing application doesn't expect to not get a return value from the target so it just spins forever until it times out and fails. The solution is to not send AERs over the wire with the fuzzer. Signed-off-by: Seth Howell <seth.howell@intel.com> Change-Id: I24fc890dfc05601953552c98690950d0981d70fc Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3739 Tested-by: SPDK CI Jenkins <sys_sgci@intel.com> Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com> Reviewed-by: Jim Harris <james.r.harris@intel.com> Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com> Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com> |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| example.json | ||
| Makefile | ||
| nvme_fuzz.c | ||
| README.md | ||
Overview
This application is intended to fuzz test the NVMe-oF target or a physical NVMe drive by submitting randomized NVMe commands through the SPDK NVMe initiator. Both local and remote drives are configured through a .ini style config file (See the -C option on the application). Multiple controllers and namespaces can be exposed to the fuzzer at a time. In order to handle multiple namespaces, the fuzzer will round robin assign a thread to each namespace and submit commands to that thread at a set queue depth. (currently 128 for I/O, 16 for Admin). The application will terminate under three conditions:
- The user specified run time expires (see the -t flag).
- One of the target controllers stops completing I/O operations back to the fuzzer i.e. controller timeout.
- The user specified a json file containing operations to run and the fuzzer has received valid completions for all of them.
Output
By default, the fuzzer will print commands that:
- Complete successfully back from the target, or
- Are outstanding at the time of a controller timeout.
Commands are dumped as named objects in json format which can then be supplied back to the
script for targeted debugging on a subsequent run. See Debugging below.
By default no output is generated when a specific command is returned with a failed status.
This can be overridden with the -V flag. if -V is specified, each command will be dumped as
it is completed in the JSON format specified above.
At the end of each test run, a summary is printed for each namespace in the following format:
NS: 0x200079262300 admin qp, Total commands completed: 462459, total successful commands: 1960, random_seed: 4276918833
Debugging
If a controller hangs when processing I/O generated by the fuzzer, the fuzzer will stop submitting I/O and dump out all outstanding I/O on the qpair that timed out. The I/O are dumped as valid json. You can combine the dumped commands from the fuzzer into a json array in a file and then pass them to the fuzzer using the -j option. Please see the example.json file in this directory for an example of a properly formed array of command structures.
Please note that you can also craft your own custom command values by using the output from the fuzzer as a template.
JSON Format
Most of the variables in the spdk_nvme_cmd structure are represented as numbers in JSON. The only exception to this rule is the dptr union. This is a 16 byte union structure that is represented as a base64 string. If writing custom commands for input, please note this distinction or the application will be unable to load your custom input.
Happy Fuzzing!