There is heap-use-after-free error when detaching a controller
when "io_path_stat" option set as true.
(if build spdk without asan ubsan, error is free(): corrupted
unsorted chunks)
It's because io_path is accessed in bdev_nvme_io_complete_nvme_status
after the io_path is freed.
io_path is freed when we detach the controller in function
_bdev_nvme_delete_io_path, this function will execute 1 and 2.
And before 4 is executed, 3 may be executed which accesses io_path.
1.spdk_put_io_channel() is called. bdev_nvme_destroy_ctrlr_channel_cb
has not been called.
2.free(io_path->stat); free(io_path);
3.bdev_nvme_poll; nbdev_io1 is success; bdev_nvme_io_complete_nvme_status()
access nbdev_io1->io_path.
4.bdev_nvme_destroy_ctrlr_channel_cb disconnect qpair and abort nbdev_io1.
This patch fixed this by moving 2 down under 4. We don't free io_path in
_bdev_nvme_delete_io_path but just remove from the nbdev_ch->io_path_list.
The processes to reproduce the error:
target: run nvmf_tgt
initiator: (build spdk with asan,ubsan enabled)
sudo ./build/examples/bdevperf --json bdevperf-multipath-rdma-active-active.json -r tmp.sock -q 128 -o 4096 -w randrw -M 50 -t 120
sudo ./scripts/rpc.py -s tmp.sock bdev_nvme_detach_controller -t rdma -a 10.10.10.10 -f IPv4 -s 4420 -n nqn.2016-06.io.spdk:cnode1 NVMe0
========
bdevperf-multipath-rdma-active-active.json
{
"subsystems": [
{
"subsystem": "bdev",
"config": [
{
"method":"bdev_nvme_attach_controller",
"params": {
"name": "NVMe0",
"trtype": "tcp",
"traddr": "10.169.204.201",
"trsvcid": "4420",
"subnqn": "nqn.2016-06.io.spdk:cnode1",
"hostnqn": "nqn.2016-06.io.spdk:init",
"adrfam": "IPv4"
}
},
{
"method":"bdev_nvme_attach_controller",
"params": {
"name": "NVMe0",
"trtype": "rdma",
"traddr": "10.10.10.10",
"trsvcid": "4420",
"subnqn": "nqn.2016-06.io.spdk:cnode1",
"hostnqn": "nqn.2016-06.io.spdk:init",
"adrfam": "IPv4",
"multipath": "multipath"
}
},
{
"method":"bdev_nvme_set_multipath_policy",
"params": {
"name": "NVMe0n1",
"policy": "active_active"
}
},
{
"method":"bdev_nvme_set_options",
"params": {
"io_path_stat": true
}
}
]
}
]
}
======
Change-Id: I8f4f9dc7195f49992a5ba9798613b64d44266e5e
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17581
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2a0b0ba782