Kernel/linux-next
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git synced 2025-01-06 14:05:39 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

io_uring: set TASK_RUNNING right after schedule

Browse Source 
Instead of constantly watching that the state of the task is running
before executing tw or taking locks in io_cqring_wait(), switch it back
to TASK_RUNNING immediately.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/246dddee247d89fd52023f785ed17cc34962a008.1672916894.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Pavel Begunkov 2023-01-05 11:22:27 +00:00 committed by Jens Axboe
parent 490c00eb4f
commit 326a9e482e
1 changed files with 2 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
io_uring/io_uring.c
View File

@ -2544,6 +2544,7 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
ret = io_cqring_wait_schedule(ctx, &iowq, &timeout); ret = io_cqring_wait_schedule(ctx, &iowq, &timeout);
if (ret < 0) if (ret < 0)
break; break;
__set_current_state(TASK_RUNNING);
/* /*
* Run task_work after scheduling and before io_should_wake(). * Run task_work after scheduling and before io_should_wake().
* If we got woken because of task_work being processed, run it * If we got woken because of task_work being processed, run it
@ -2556,10 +2557,8 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
check_cq = READ_ONCE(ctx->check_cq); check_cq = READ_ONCE(ctx->check_cq);
if (unlikely(check_cq)) { if (unlikely(check_cq)) {
/* let the caller flush overflows, retry */ /* let the caller flush overflows, retry */
if (check_cq & BIT(IO_CHECK_CQ_OVERFLOW_BIT)) { if (check_cq & BIT(IO_CHECK_CQ_OVERFLOW_BIT))
finish_wait(&ctx->cq_wait, &iowq.wq);
io_cqring_do_overflow_flush(ctx); io_cqring_do_overflow_flush(ctx);
}
if (check_cq & BIT(IO_CHECK_CQ_DROPPED_BIT)) { if (check_cq & BIT(IO_CHECK_CQ_DROPPED_BIT)) {
ret = -EBADR; ret = -EBADR;
break; break;