Kernel/linux
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2025-01-10 07:10:27 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

io_uring: add likely/unlikely in io_get_sqring()

Browse Source 
The number of SQEs to submit is specified by a user, so io_get_sqring()
in most of the cases succeeds. Hint compilers about that.

Checking ASM genereted by gcc 9.2.0 for x64, there is one branch
misprediction.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Pavel Begunkov 2019-11-21 21:24:56 +03:00 committed by Jens Axboe
parent d732447fed
commit 9835d6fafb
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
fs/io_uring.c
View File

@ -3129,11 +3129,11 @@ static bool io_get_sqring(struct io_ring_ctx *ctx, struct sqe_submit *s)
*/ */
head = ctx->cached_sq_head; head = ctx->cached_sq_head;
/* make sure SQ entry isn't read before tail */ /* make sure SQ entry isn't read before tail */
if (head == smp_load_acquire(&rings->sq.tail)) if (unlikely(head == smp_load_acquire(&rings->sq.tail)))
return false; return false;
head = READ_ONCE(sq_array[head & ctx->sq_mask]); head = READ_ONCE(sq_array[head & ctx->sq_mask]);
if (head < ctx->sq_entries) { if (likely(head < ctx->sq_entries)) {
s->ring_file = NULL; s->ring_file = NULL;
s->sqe = &ctx->sq_sqes[head]; s->sqe = &ctx->sq_sqes[head];
s->sequence = ctx->cached_sq_head; s->sequence = ctx->cached_sq_head;