Kernel/linux-next
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git synced 2024-12-29 17:22:07 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

io_uring: don't vmap single page regions

Browse Source 
When io_check_coalesce_buffer() meets a single page buffer it bails out
and tells that it can be coalesced. That's fine for registered buffers
as io_coalesce_buffer() wouldn't change anything, but the region code
now uses the function to decided on whether to vmap the buffer or not.

Report that a single page buffer is trivially coalescable and let
io_sqe_buffer_register() to filter them.

Fixes: cc2f1a864c ("io_uring/memmap: optimise single folio regions")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/cb83e053f318857068447d40c95becebcd8aeced.1733689833.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Pavel Begunkov 2024-12-08 21:46:01 +00:00 committed by Jens Axboe
parent 02a6a5eb59
commit 56c71fe7f3
1 changed files with 5 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
io_uring/rsrc.c
View File

@ -675,14 +675,9 @@ bool io_check_coalesce_buffer(struct page **page_array, int nr_pages,
unsigned int count = 1, nr_folios = 1; unsigned int count = 1, nr_folios = 1;
int i; int i;
if (nr_pages <= 1)
return false;
data->nr_pages_mid = folio_nr_pages(folio); data->nr_pages_mid = folio_nr_pages(folio);
if (data->nr_pages_mid == 1)
return false;
data->folio_shift = folio_shift(folio); data->folio_shift = folio_shift(folio);
/* /*
* Check if pages are contiguous inside a folio, and all folios have * Check if pages are contiguous inside a folio, and all folios have
* the same page count except for the head and tail. * the same page count except for the head and tail.
@ -750,8 +745,10 @@ static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx,
} }
/* If it's huge page(s), try to coalesce them into fewer bvec entries */ /* If it's huge page(s), try to coalesce them into fewer bvec entries */
if (io_check_coalesce_buffer(pages, nr_pages, &data)) if (nr_pages > 1 && io_check_coalesce_buffer(pages, nr_pages, &data)) {
if (data.nr_pages_mid != 1)
coalesced = io_coalesce_buffer(&pages, &nr_pages, &data); coalesced = io_coalesce_buffer(&pages, &nr_pages, &data);
}
imu = kvmalloc(struct_size(imu, bvec, nr_pages), GFP_KERNEL); imu = kvmalloc(struct_size(imu, bvec, nr_pages), GFP_KERNEL);
if (!imu) if (!imu)