ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9bff828f99
Spdk/scripts
History
Richael Zhuang 9bff828f99 sock: introduce dynamic zerocopy according to data size 
MSG_ZEROCOPY is not always effective as mentioned in
https://www.kernel.org/doc/html/v4.15/networking/msg_zerocopy.html.

Currently in spdk, once we enable sendmsg zerocopy, then all data
transferred through _sock_flush are sent with zerocopy, and vice
versa. Here dynamic zerocopy is introduced to allow data sent with
MSG_ZEROCOPY or not according to its size, which can be enabled by
setting "enable_dynamic_zerocopy" as true.

Test with 16 P4610 NVMe SSD, 2 initiators, target's and initiators'
configurations are the same as spdk report:
https://ci.spdk.io/download/performance-reports/SPDK_tcp_perf_report_2104.pdf

For posix socket, rw_percent=0(randwrite), it has 1.9%~8.3% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it has no obvious
influence when read percentage is greater than 50%.

For uring socket, rw_percent=0(randwrite), it has 1.8%~7.9% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it still has
1%~7% improvement when read percentage is greater than 50%.

The following is part of the detailed data.

posix:
qdepth=128
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	286.5	298.5	4.19%		 307	304.15	-0.93%
4	1042.5	1107	6.19%		1135.5	1136	0.04%
8	1952.5	2058	5.40%		2170.5	2170.5	0.00%
12	2658.5	2879	8.29%		3042	3046	0.13%
16	3247.5	3460.5	6.56%		3793.5	3775	-0.49%
24	4232.5	4459.5	5.36%		4614.5	4756.5	3.08%
32	4810	5095	5.93%		4488	4845	7.95%
40	5306.5	5435	2.42%		4427.5	4902	10.72%

qdepth=512
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1    275	 287	4.36%		294.4	295.45	0.36%
4	 979	1041	6.33%		1073	1083.5	0.98%
8	1822.5	1914.5	5.05%		2030.5	2018.5	-0.59%
12	2441	2598.5	6.45%		2808.5	2779.5	-1.03%
16	2920.5	3109.5	6.47%		3455	3411.5	-1.26%
24	3709	3972.5	7.10%		4483.5	4502.5	0.42%
32	4225.5	4532.5	7.27%		4463.5	4733	6.04%
40	4790.5	4884.5	1.96%		4427	4904.5	10.79%

uring:
qdepth=128
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	270.5	287.5	6.28%		295.75	304.75	3.04%
4	1018.5	1089.5	6.97%		1119.5	1156.5	3.31%
8	1907	2055	7.76%		2127	2211.5	3.97%
12	2614	2801	7.15%		2982.5	3061.5	2.65%
16	3169.5	3420	7.90%		3654.5	3781.5	3.48%
24	4109.5	4414	7.41%		4691.5	4750.5	1.26%
32	4752.5	4908	3.27%		4494	4825.5	7.38%
40	5233.5	5327	1.79%		4374.5	4891	11.81%

qdepth=512
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	259.95	 276	6.17%		286.65	294.8	2.84%
4	955 	1021	6.91%		1070.5	1100	2.76%
8	1772	1903.5	7.42%		1992.5	2077.5	4.27%
12	2380.5	2543.5	6.85%		2752.5	2860	3.91%
16	2920.5	3099	6.11%		3391.5	3540	4.38%
24	3697	3912	5.82%		4401	4637	5.36%
32	4256.5	4454.5	4.65%		4516	4777	5.78%
40	4707	4968.5	5.56%		4400.5	4933	12.10%

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I730dcf89ed2bf3efe91586421a89045fc11c81f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12210
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-28 07:29:28 +00:00
..
bash-completion scripts/bash-completion: Use default filename completion for ./configure 2021-12-14 09:09:24 +00:00
bpf scripts/bpf: remove bdev_nvme_destroy_qpair probe 2022-04-20 08:20:02 +00:00
ceph spelling: scripts 2021-12-03 08:13:04 +00:00
perf scripts: fix PYTHONPATH references to scripts/ directory 2022-04-18 17:22:09 +00:00
pkgdep pkgdep/centos: fix centos routines for installing the packages 2022-04-26 07:47:27 +00:00
vagrant scripts/vagrant: add missing Fedora 35 image name 2022-04-13 14:41:49 +00:00
arm_cross_compile.sh scripts: ARM64 Cross-Compilation Support 2021-09-09 20:26:59 +00:00
bpftrace.sh scripts/bpftrace: use SO names in bpftrace scripts 2021-10-20 23:39:59 +00:00
check_format.sh check_format.sh: only use specific v3.1.0 shfmt version 2022-04-27 07:36:14 +00:00
common.sh scripts/common: set special name to test partitions 2022-01-26 19:07:43 +00:00
core-collector.sh scripts/core-collector: Don't limit core dump collection 2022-01-05 09:24:22 +00:00
detect_cc.sh check_format: Reformat the Bash code in compliance with shfmt 2020-05-07 20:52:21 +00:00
dpdk_mem_info.py scripts: add a parser for the dpdk memory stats. 2020-01-24 05:46:43 +00:00
eofnl scripts/check_format: check trailing whitespace 2017-10-24 17:35:23 -04:00
fio-wrapper test/nvmf: enable fio verify_backlog option 2022-02-04 21:58:18 +00:00
gdb_macros.py scripts: Use range instead of xrange 2019-02-13 22:54:15 +00:00
gen_ftl.sh check_format: Reformat the Bash code in compliance with shfmt 2020-05-07 20:52:21 +00:00
gen_nvme.sh scripts/gen_nvme.sh: add option for remote bdev JSON configuration 2021-09-24 07:39:41 +00:00
genconfig.py genconfig.py: allow numbers in config option names 2020-02-20 14:40:19 +00:00
get-pmr pkgdep/git: Bump vanilla qemu to v6.0.0 2021-05-11 11:51:01 +00:00
histogram.py histograms: add script for parsing histogram data 2019-01-24 14:14:32 +00:00
iostat.py scripts: move python modules to python directory 2022-04-05 14:40:47 +00:00
lspci scripts/rpc: option to format framework_get_pci_devices for use with lspci 2021-12-14 09:08:59 +00:00
pc_libs.sh build: generate pkg-config files for SPDK 2021-01-21 09:46:32 +00:00
pc_modules.sh build: generate pkg-config files for SPDK 2021-01-21 09:46:32 +00:00
pc.sh build: install generated pkg-config files 2021-03-17 11:10:45 +00:00
pkgdep.sh pkgdep: Fix ID lookup 2021-11-18 08:17:50 +00:00
posix.txt build: Automatically detect use of POSIX includes 2017-05-08 13:20:37 -07:00
prep_benchmarks.sh check_format: Reformat the Bash code in compliance with shfmt 2020-05-07 20:52:21 +00:00
qat_setup.sh qat_setup: Don't use absolute path to igb_uio driver 2021-12-02 08:08:10 +00:00
rpc_http_proxy.py scripts: move python modules to python directory 2022-04-05 14:40:47 +00:00
rpc.py sock: introduce dynamic zerocopy according to data size 2022-04-28 07:29:28 +00:00
rxe_cfg_small.sh test/nvmf: Use rxe_cfg to get rxe-enabled net devices 2021-04-12 09:41:20 +00:00
setup.sh scripts/setup: Skip devices which have any valid data present 2022-04-06 07:21:17 +00:00
spdk-gpt.py scripts/spdk-gpt: Fix path lookup of the target device 2022-01-25 08:13:40 +00:00
spdkcli.py scripts: move python modules to python directory 2022-04-05 14:40:47 +00:00
sync_dev_uevents.sh sync_dev_uevents: Break from the loop if kernel seqnum matches mdev's 2021-02-04 16:10:49 +00:00