Kernel/linux-stable
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git synced 2025-01-06 13:16:22 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
Linux kernel stable tree
916,565 Commits 106 Branches 5,184 Tags 6 GiB
C 97.5%
Assembly 1%
Shell 0.6%
Python 0.3%
Makefile 0.3%
412895f03c
Go to file
Roman Penyaev 412895f03c epoll: atomically remove wait entry on wake up 
This patch does two things:

 - fixes a lost wakeup introduced by commit 339ddb53d3 ("fs/epoll:
   remove unnecessary wakeups of nested epoll")

 - improves performance for events delivery.

The description of the problem is the following: if N (>1) threads are
waiting on ep->wq for new events and M (>1) events come, it is quite
likely that >1 wakeups hit the same wait queue entry, because there is
quite a big window between __add_wait_queue_exclusive() and the
following __remove_wait_queue() calls in ep_poll() function.

This can lead to lost wakeups, because thread, which was woken up, can
handle not all the events in ->rdllist.  (in better words the problem is
described here: https://lkml.org/lkml/2019/10/7/905)

The idea of the current patch is to use init_wait() instead of
init_waitqueue_entry().

Internally init_wait() sets autoremove_wake_function as a callback,
which removes the wait entry atomically (under the wq locks) from the
list, thus the next coming wakeup hits the next wait entry in the wait
queue, thus preventing lost wakeups.

Problem is very well reproduced by the epoll60 test case [1].

Wait entry removal on wakeup has also performance benefits, because
there is no need to take a ep->lock and remove wait entry from the queue
after the successful wakeup.  Here is the timing output of the epoll60
test case:

  With explicit wakeup from ep_scan_ready_list() (the state of the
  code prior 339ddb53d3):

    real    0m6.970s
    user    0m49.786s
    sys     0m0.113s

 After this patch:

   real    0m5.220s
   user    0m36.879s
   sys     0m0.019s

The other testcase is the stress-epoll [2], where one thread consumes
all the events and other threads produce many events:

  With explicit wakeup from ep_scan_ready_list() (the state of the
  code prior 339ddb53d3):

    threads  events/ms  run-time ms
          8       5427         1474
         16       6163         2596
         32       6824         4689
         64       7060         9064
        128       6991        18309

 After this patch:

    threads  events/ms  run-time ms
          8       5598         1429
         16       7073         2262
         32       7502         4265
         64       7640         8376
        128       7634        16767

 (number of "events/ms" represents event bandwidth, thus higher is
  better; number of "run-time ms" represents overall time spent
  doing the benchmark, thus lower is better)

[1] tools/testing/selftests/filesystems/epoll/epoll_wakeup_test.c
[2] https://github.com/rouming/test-tools/blob/master/stress-epoll.c

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Jason Baron <jbaron@akamai.com>
Cc: Khazhismel Kumykov <khazhy@google.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Heiher <r@hev.cc>
Cc: <stable@vger.kernel.org>
Link: http://lkml.kernel.org/r/20200430130326.1368509-2-rpenyaev@suse.de
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-05-07 19:27:21 -07:00
arch arch/x86/kvm/svm/sev.c: change flag passed to GUP fast in sev_pin_memory() 2020-05-07 19:27:20 -07:00
block block: remove the bd_openers checks in blk_drop_partitions 2020-04-30 10:25:43 -06:00
certs .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
crypto Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2020-04-01 14:47:40 -07:00
Documentation Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-05-06 20:53:22 -07:00
drivers Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-05-06 20:53:22 -07:00
fs epoll: atomically remove wait entry on wake up 2020-05-07 19:27:21 -07:00
include Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-05-06 20:53:22 -07:00
init Kbuild updates for v5.7 (2nd) 2020-04-11 09:46:12 -07:00
ipc ipc/mqueue.c: change __do_notify() to bypass check_kill_permission() 2020-05-07 19:27:20 -07:00
kernel kernel/kcov.c: fix typos in kcov_remote_start documentation 2020-05-07 19:27:20 -07:00
lib linux-kselftest-kunit-5.7-rc4 2020-04-30 16:32:47 -07:00
LICENSES LICENSES: Rename other to deprecated 2019-05-03 06:34:32 -06:00
mm percpu: make pcpu_alloc() aware of current gfp context 2020-05-07 19:27:21 -07:00
net Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-05-06 20:53:22 -07:00
samples vmalloc: fix remap_vmalloc_range() bounds checks 2020-04-21 11:11:56 -07:00
scripts scripts/gdb: repair rb_first() and rb_last() 2020-05-07 19:27:20 -07:00
security selinux/stable-5.7 PR 20200430 2020-04-30 16:35:45 -07:00
sound sound fixes for 5.7-rc4 2020-05-01 11:05:28 -07:00
tools kselftests: introduce new epoll60 testcase for catching lost wakeups 2020-05-07 19:27:21 -07:00
usr kbuild: fix comment about missing include guard detection 2020-04-11 12:09:48 +09:00
virt KVM: Pass kvm_init()'s opaque param to additional arch funcs 2020-03-31 10:48:03 -04:00
.clang-format clang-format: Update with the latest for_each macro list 2020-04-18 13:49:33 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
.mailmap mailmap: Add Sedat Dilek (replacement for expired email address) 2020-04-11 09:28:34 -07:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Hand MIPS over to Thomas 2020-02-24 22:43:18 -08:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig docs: kbuild: convert docs to ReST and rename to *.rst 2019-06-14 14:21:21 -06:00
MAINTAINERS Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-05-06 20:53:22 -07:00
Makefile Linux 5.7-rc4 2020-05-03 14:56:04 -07:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.