f8cf17f58e
Something changed about the way we support AERs in the target. Now, the AER commands remain queued in the target side causing the fuzzer to think it is timed out. Signed-off-by: Seth Howell <seth.howell@intel.com> Change-Id: I8e892a11d9922e84cd7a09e6adc8b69b6db4e8d2 Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3453 Tested-by: SPDK CI Jenkins <sys_sgci@intel.com> Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com> Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com> Reviewed-by: Karol Latecki <karol.latecki@intel.com> Community-CI: Mellanox Build Bot |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| example.json | ||
| Makefile | ||
| nvme_fuzz.c | ||
| README.md | ||
Overview
This application is intended to fuzz test the NVMe-oF target or a physical NVMe drive by submitting randomized NVMe commands through the SPDK NVMe initiator. Both local and remote drives are configured through a .ini style config file (See the -C option on the application). Multiple controllers and namespaces can be exposed to the fuzzer at a time. In order to handle multiple namespaces, the fuzzer will round robin assign a thread to each namespace and submit commands to that thread at a set queue depth. (currently 128 for I/O, 16 for Admin). The application will terminate under three conditions:
- The user specified run time expires (see the -t flag).
- One of the target controllers stops completing I/O operations back to the fuzzer i.e. controller timeout.
- The user specified a json file containing operations to run and the fuzzer has received valid completions for all of them.
Output
By default, the fuzzer will print commands that:
- Complete successfully back from the target, or
- Are outstanding at the time of a controller timeout.
Commands are dumped as named objects in json format which can then be supplied back to the
script for targeted debugging on a subsequent run. See Debugging below.
By default no output is generated when a specific command is returned with a failed status.
This can be overridden with the -V flag. if -V is specified, each command will be dumped as
it is completed in the JSON format specified above.
At the end of each test run, a summary is printed for each namespace in the following format:
NS: 0x200079262300 admin qp, Total commands completed: 462459, total successful commands: 1960, random_seed: 4276918833
Debugging
If a controller hangs when processing I/O generated by the fuzzer, the fuzzer will stop submitting I/O and dump out all outstanding I/O on the qpair that timed out. The I/O are dumped as valid json. You can combine the dumped commands from the fuzzer into a json array in a file and then pass them to the fuzzer using the -j option. Please see the example.json file in this directory for an example of a properly formed array of command structures.
Please note that you can also craft your own custom command values by using the output from the fuzzer as a template.
JSON Format
Most of the variables in the spdk_nvme_cmd structure are represented as numbers in JSON. The only exception to this rule is the dptr union. This is a 16 byte union structure that is represented as a base64 string. If writing custom commands for input, please note this distinction or the application will be unable to load your custom input.
Happy Fuzzing!