ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
d11edb7ac8
Spdk/test/unit/lib
History
Richael Zhuang 9bff828f99 sock: introduce dynamic zerocopy according to data size 
MSG_ZEROCOPY is not always effective as mentioned in
https://www.kernel.org/doc/html/v4.15/networking/msg_zerocopy.html.

Currently in spdk, once we enable sendmsg zerocopy, then all data
transferred through _sock_flush are sent with zerocopy, and vice
versa. Here dynamic zerocopy is introduced to allow data sent with
MSG_ZEROCOPY or not according to its size, which can be enabled by
setting "enable_dynamic_zerocopy" as true.

Test with 16 P4610 NVMe SSD, 2 initiators, target's and initiators'
configurations are the same as spdk report:
https://ci.spdk.io/download/performance-reports/SPDK_tcp_perf_report_2104.pdf

For posix socket, rw_percent=0(randwrite), it has 1.9%~8.3% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it has no obvious
influence when read percentage is greater than 50%.

For uring socket, rw_percent=0(randwrite), it has 1.8%~7.9% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it still has
1%~7% improvement when read percentage is greater than 50%.

The following is part of the detailed data.

posix:
qdepth=128
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	286.5	298.5	4.19%		 307	304.15	-0.93%
4	1042.5	1107	6.19%		1135.5	1136	0.04%
8	1952.5	2058	5.40%		2170.5	2170.5	0.00%
12	2658.5	2879	8.29%		3042	3046	0.13%
16	3247.5	3460.5	6.56%		3793.5	3775	-0.49%
24	4232.5	4459.5	5.36%		4614.5	4756.5	3.08%
32	4810	5095	5.93%		4488	4845	7.95%
40	5306.5	5435	2.42%		4427.5	4902	10.72%

qdepth=512
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1    275	 287	4.36%		294.4	295.45	0.36%
4	 979	1041	6.33%		1073	1083.5	0.98%
8	1822.5	1914.5	5.05%		2030.5	2018.5	-0.59%
12	2441	2598.5	6.45%		2808.5	2779.5	-1.03%
16	2920.5	3109.5	6.47%		3455	3411.5	-1.26%
24	3709	3972.5	7.10%		4483.5	4502.5	0.42%
32	4225.5	4532.5	7.27%		4463.5	4733	6.04%
40	4790.5	4884.5	1.96%		4427	4904.5	10.79%

uring:
qdepth=128
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	270.5	287.5	6.28%		295.75	304.75	3.04%
4	1018.5	1089.5	6.97%		1119.5	1156.5	3.31%
8	1907	2055	7.76%		2127	2211.5	3.97%
12	2614	2801	7.15%		2982.5	3061.5	2.65%
16	3169.5	3420	7.90%		3654.5	3781.5	3.48%
24	4109.5	4414	7.41%		4691.5	4750.5	1.26%
32	4752.5	4908	3.27%		4494	4825.5	7.38%
40	5233.5	5327	1.79%		4374.5	4891	11.81%

qdepth=512
rw_percent      0             |           30
cpu  origin  thisPatch  opt   | origin  thisPatch opt
1	259.95	 276	6.17%		286.65	294.8	2.84%
4	955 	1021	6.91%		1070.5	1100	2.76%
8	1772	1903.5	7.42%		1992.5	2077.5	4.27%
12	2380.5	2543.5	6.85%		2752.5	2860	3.91%
16	2920.5	3099	6.11%		3391.5	3540	4.38%
24	3697	3912	5.82%		4401	4637	5.36%
32	4256.5	4454.5	4.65%		4516	4777	5.78%
40	4707	4968.5	5.56%		4400.5	4933	12.10%

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I730dcf89ed2bf3efe91586421a89045fc11c81f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12210
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-04-28 07:29:28 +00:00
..
accel accel: cleanup by getting rid of capabilties enum 2022-04-14 08:32:50 +00:00
bdev bdev/nvme: Fix aborting fuse commands 2022-04-26 07:47:09 +00:00
blob blob_ut: bs_opts initialized with wrong size 2022-02-02 08:25:02 +00:00
blobfs test: optimize unit test .gitignore files 2021-08-20 07:29:57 +00:00
dma dma: Update memory domain context structure 2021-10-20 22:55:52 +00:00
env_dpdk test: optimize unit test .gitignore files 2021-08-20 07:29:57 +00:00
event event: Add a user option to change the size of spdk_msg_mempool 2022-03-09 08:00:28 +00:00
ftl spelling: test 2021-12-03 08:13:22 +00:00
idxd idxd: Make many internal idxd_user functions take an idxd_user object 2022-04-20 08:20:45 +00:00
init spelling: test 2021-12-03 08:13:22 +00:00
ioat test: optimize unit test .gitignore files 2021-08-20 07:29:57 +00:00
iscsi ut: Use unit/lib/json_mock.c for stubs 2022-01-31 09:44:28 +00:00
json spelling: test 2021-12-03 08:13:22 +00:00
jsonrpc spelling: test 2021-12-03 08:13:22 +00:00
log test: optimize unit test .gitignore files 2021-08-20 07:29:57 +00:00
lvol lvol: lvol_get_xattr_value failure undetectable 2022-01-31 09:44:16 +00:00
notify test: optimize unit test .gitignore files 2021-08-20 07:29:57 +00:00
nvme nvme_rdma: Wait until lingering qpair becomes quiet before completing disconnection 2022-04-18 18:35:29 +00:00
nvmf nvmf/tcp: delay qpair destruction 2022-04-25 07:36:05 +00:00
reduce lib/reduce: Copy user's buffers if SGL is not supported 2022-03-31 09:34:52 +00:00
rpc ut: Use unit/lib/json_mock.c for stubs 2022-01-31 09:44:28 +00:00
scsi Lib/iSCSI: add the LUN Resize support 2022-01-20 07:56:23 +00:00
sock sock: introduce dynamic zerocopy according to data size 2022-04-28 07:29:28 +00:00
thread event: Add a user option to change the size of spdk_msg_mempool 2022-03-09 08:00:28 +00:00
util dif: enhance copy API to support block-aligned bounce_iov 2022-04-01 08:29:12 +00:00
vhost lib/vhost: move vhost_user related fields from spdk_vhost_dev 2022-04-15 07:49:32 +00:00
json_mock.c ut/json_mock: Add stubs for json_write_uint8 and _uint16 2022-01-31 09:44:28 +00:00
Makefile unittest/rpc: add rpc unittest 2021-09-06 09:55:33 +00:00