ivampiresp/Spdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

nvmf: enable dynamic buf_cache_size calculation

Browse Source 
Allow transports to specify a default UINT32_MAX
as the buf_cache_size. If user does not override this
when creating the transport, calculate the buf_cache_size
dynamically using the number of poll groups and the
size of the buffer pool (num_shared_buffers).  We will
allocate 75% of the buffers for the caches, meaning
the buf_cache_size will be calculated as:

(num_shared_buffers * 3 / 4) / num_poll_groups

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I97768aea701060bbe0ff1925e5322229fa8d051c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17334
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
This commit is contained in:
Jim Harris 2023-03-24 22:26:07 +00:00
parent 280a3abc9c
commit 3092c61d26
1 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
lib/nvmf/transport.c
View File

@ -557,6 +557,24 @@ nvmf_transport_poll_group_create(struct spdk_nvmf_transport *transport,
}
tgroup->buf_cache_size = transport->opts.buf_cache_size;
/* buf_cache_size of UINT32_MAX means the value should be calculated dynamically
* based on the number of buffers in the shared pool and the number of poll groups
* that are sharing them. We allocate 75% of the pool for the cache, and then
* divide that by number of poll groups to determine the buf_cache_size for this
* poll group.
*/
if (tgroup->buf_cache_size == UINT32_MAX) {
uint32_t num_shared_buffers = transport->opts.num_shared_buffers;
/* Theoretically the nvmf library can dynamically add poll groups to
* the target, after transports have already been created. We aren't
* going to try to really handle this case efficiently, just do enough
* here to ensure we don't divide-by-zero.
*/
uint16_t num_poll_groups = group->tgt->num_poll_groups ? : spdk_env_get_core_count();
tgroup->buf_cache_size = (num_shared_buffers * 3 / 4) / num_poll_groups;
}
bufs = calloc(tgroup->buf_cache_size, sizeof(struct spdk_nvmf_transport_pg_cache_buf *));
if (!bufs) {