linux-stable/kernel
Tejun Heo 6a010a49b6 cgroup: Make !percpu threadgroup_rwsem operations optional
3942a9bd7b ("locking, rcu, cgroup: Avoid synchronize_sched() in
__cgroup_procs_write()") disabled percpu operations on threadgroup_rwsem
because the impiled synchronize_rcu() on write locking was pushing up the
latencies too much for android which constantly moves processes between
cgroups.

This makes the hotter paths - fork and exit - slower as they're always
forced into the slow path. There is no reason to force this on everyone
especially given that more common static usage pattern can now completely
avoid write-locking the rwsem. Write-locking is elided when turning on and
off controllers on empty sub-trees and CLONE_INTO_CGROUP enables seeding a
cgroup without grabbing the rwsem.

Restore the default percpu operations and introduce the mount option
"favordynmods" and config option CGROUP_FAVOR_DYNMODS for users who need
lower latencies for the dynamic operations.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Michal Koutn� <mkoutny@suse.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Dmitry Shmidt <dimitrysh@google.com>
Cc: Oleg Nesterov <oleg@redhat.com>
2022-07-23 04:29:02 -10:00
..
bpf Networking fixes for 5.19-rc1, including fixes from bpf, and netfilter. 2022-06-02 12:50:16 -07:00
cgroup cgroup: Make !percpu threadgroup_rwsem operations optional 2022-07-23 04:29:02 -10:00
configs x86/configs: Add x86 debugging Kconfig fragment plus docs 2022-04-06 19:56:29 +02:00
debug Modules updates for v5.19-rc1 2022-05-26 17:13:43 -07:00
dma swiotlb: fix setting ->force_bounce 2022-06-02 07:17:59 +02:00
entry Locking changes in this cycle were: 2022-05-24 10:18:23 -07:00
events Two small perf updates: 2022-06-05 10:40:31 -07:00
futex drm for 5.19-rc1 2022-05-25 16:18:27 -07:00
gcov gcov: Remove compiler version check 2021-12-02 17:25:21 +09:00
irq Updates for interrupt core and drivers: 2022-05-23 16:58:49 -07:00
kcsan linux-kselftest-kunit-5.19-rc1 2022-05-25 11:32:53 -07:00
livepatch Livepatching changes for 5.19 2022-06-02 08:55:01 -07:00
locking sysctl changes for v5.19-rc1 2022-05-26 16:57:20 -07:00
module module: Fix prefix for module.sig_enforce module param 2022-06-02 12:44:33 -07:00
power cxl for 5.19 2022-05-27 21:24:19 -07:00
printk Revert "printk: wake up all waiters" 2022-05-27 13:04:46 +02:00
rcu sysctl changes for v5.19-rc1 2022-05-26 16:57:20 -07:00
sched psi: dont alloc memory for psi by default 2022-06-07 07:11:47 -10:00
time While looking at the ptrace problems with PREEMPT_RT and the problems 2022-06-03 16:13:25 -07:00
trace arm64 fixes for 5.19-rc1: 2022-06-03 14:05:34 -07:00
.gitignore
acct.c kernel/acct: move acct sysctls to its own file 2022-04-06 13:43:44 -07:00
async.c Revert "module, async: async_synchronize_full() on module init iff async is used" 2022-02-03 11:20:34 -08:00
audit_fsnotify.c fsnotify: make allow_dups a property of the group 2022-04-25 14:37:18 +02:00
audit_tree.c audit: use fsnotify group lock helpers 2022-04-25 14:37:28 +02:00
audit_watch.c fsnotify: pass flags argument to fsnotify_alloc_group() 2022-04-25 14:37:12 +02:00
audit.c audit: improve audit queue handling when "audit=1" on cmdline 2022-01-25 13:22:51 -05:00
audit.h audit: log AUDIT_TIME_* records only from rules 2022-02-22 13:51:40 -05:00
auditfilter.c audit/stable-5.17 PR 20220110 2022-01-11 13:08:21 -08:00
auditsc.c audit,io_uring,io-wq: call __audit_uring_exit for dummy contexts 2022-05-17 15:03:36 -04:00
backtracetest.c
bounds.c
capability.c xfs: don't generate selinux audit messages for capability testing 2022-03-09 10:32:06 -08:00
cfi.c cfi: Use rcu_read_{un}lock_sched_notrace 2021-08-11 13:11:12 -07:00
compat.c arch: remove compat_alloc_user_space 2021-09-08 15:32:35 -07:00
configs.c
context_tracking.c
cpu_pm.c PM: cpu: Make notifier chain use a raw_spinlock_t 2021-08-16 18:55:32 +02:00
cpu.c Intel Trust Domain Extensions 2022-05-23 17:51:12 -07:00
crash_core.c Not a lot of material this cycle. Many singleton patches against various 2022-05-27 11:22:03 -07:00
crash_dump.c
cred.c x86: Mark __invalid_creds() __noreturn 2022-03-15 10:32:44 +01:00
delayacct.c delayacct: track delays from write-protect copy 2022-06-01 15:55:25 -07:00
dma.c
exec_domain.c
exit.c ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
extable.c lkdtm: Really write into kernel text in WRITE_KERN 2022-02-16 23:25:12 +11:00
fail_function.c
fork.c This set of changes updates init and user mode helper tasks to be 2022-06-03 16:03:05 -07:00
freezer.c sched: Add get_current_state() 2021-06-18 11:43:08 +02:00
gen_kheaders.sh kheaders: Have cpio unconditionally replace files 2022-05-08 03:16:59 +09:00
groups.c
hung_task.c Not a lot of material this cycle. Many singleton patches against various 2022-05-27 11:22:03 -07:00
iomem.c
irq_work.c irq_work: use kasan_record_aux_stack_noalloc() record callstack 2022-04-15 14:49:55 -07:00
jump_label.c jump_label: Fix jump_label_text_reserved() vs __init 2021-07-05 10:46:20 +02:00
kallsyms.c ftrace: Add ftrace_lookup_symbols function 2022-05-10 14:42:06 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks locking/rwlock: Provide RT variant 2021-08-17 17:50:51 +02:00
Kconfig.preempt Revert "signal, x86: Delay calling signals in atomic on RT enabled kernels" 2022-03-31 10:36:55 +02:00
kcov.c kcov: update pos before writing pc in trace function 2022-05-25 13:05:42 -07:00
kexec_core.c Not a lot of material this cycle. Many singleton patches against various 2022-05-27 11:22:03 -07:00
kexec_elf.c
kexec_file.c RISC-V Patches for the 5.19 Merge Window, Part 1 2022-05-31 14:10:54 -07:00
kexec_internal.h
kexec.c kexec: avoid compat_alloc_user_space 2021-09-08 15:32:34 -07:00
kheaders.c
kmod.c
kprobes.c tracing updates for 5.19: 2022-05-29 10:31:36 -07:00
ksysfs.c kernel/ksysfs.c: use helper macro __ATTR_RW 2022-03-23 19:00:33 -07:00
kthread.c kthread: unexport kthread_blkcg 2022-05-02 14:06:20 -06:00
latencytop.c latencytop: move sysctl to its own file 2022-04-21 11:40:59 -07:00
Makefile Modules updates for v5.19-rc1 2022-05-26 17:13:43 -07:00
module_signature.c
notifier.c notifier: Add blocking/atomic_notifier_chain_register_unique_prio() 2022-05-19 19:30:30 +02:00
nsproxy.c memcg: enable accounting for new namesapces and struct nsproxy 2021-09-03 09:58:12 -07:00
padata.c padata: replace cpumask_weight with cpumask_empty in padata.c 2022-01-31 11:21:46 +11:00
panic.c sysctl changes for v5.19-rc1 2022-05-26 16:57:20 -07:00
params.c kobject: remove kset from struct kset_uevent_ops callbacks 2021-12-28 11:26:18 +01:00
pid_namespace.c kernel: pid_namespace: use NULL instead of using plain integer as pointer 2022-04-29 14:38:00 -07:00
pid.c pid: add pidfd_get_task() helper 2021-10-14 13:29:18 +02:00
profile.c exit: Remove profile_handoff_task 2022-01-08 12:43:57 -06:00
ptrace.c While looking at the ptrace problems with PREEMPT_RT and the problems 2022-06-03 16:13:25 -07:00
range.c
reboot.c More power management updates for 5.19-rc1 2022-05-30 11:37:26 -07:00
regset.c
relay.c relay: remove redundant assignment to pointer buf 2022-05-12 20:38:37 -07:00
resource_kunit.c
resource.c kernel/resource: fix kfree() of bootmem memory again 2022-03-23 19:00:35 -07:00
rseq.c rseq: Remove broken uapi field layout on 32-bit little endian 2022-02-02 13:11:34 +01:00
scftorture.c scftorture: Fix distribution of short handler delays 2022-04-11 17:07:29 -07:00
scs.c kasan, vmalloc: only tag normal vmalloc allocations 2022-03-24 19:06:48 -07:00
seccomp.c seccomp: Add wait_killable semantic to seccomp user notifier 2022-05-03 14:11:58 -07:00
signal.c While looking at the ptrace problems with PREEMPT_RT and the problems 2022-06-03 16:13:25 -07:00
smp.c Scheduler changes in this cycle were: 2022-05-24 11:11:13 -07:00
smpboot.c cpu/hotplug: Allow the CPU in CPU_UP_PREPARE state to be brought up again. 2022-04-12 14:13:01 +02:00
smpboot.h
softirq.c smp: Make softirq handling RT safe in flush_smp_call_function_queue() 2022-05-01 10:03:43 +02:00
stackleak.c stackleak: add on/off stack variants 2022-05-08 01:33:09 -07:00
stacktrace.c uaccess: remove CONFIG_SET_FS 2022-02-25 09:36:06 +01:00
static_call_inline.c static_call: Don't make __static_call_return0 static 2022-04-05 09:59:38 +02:00
static_call.c static_call: Don't make __static_call_return0 static 2022-04-05 09:59:38 +02:00
stop_machine.c Scheduler changes in this cycle were: 2022-05-24 11:11:13 -07:00
sys_ni.c mm/mempolicy: wire up syscall set_mempolicy_home_node 2022-01-15 16:30:30 +02:00
sys.c arm64/sme: Implement vector length configuration prctl()s 2022-04-22 18:50:54 +01:00
sysctl-test.c kernel/sysctl-test: Remove some casts which are no-longer required 2021-06-23 16:41:24 -06:00
sysctl.c sysctl changes for v5.19-rc1 2022-05-26 16:57:20 -07:00
task_work.c task_work: allow TWA_SIGNAL without a rescheduling IPI 2022-04-30 08:39:32 -06:00
taskstats.c kernel: make taskstats available from all net namespaces 2022-04-29 14:38:03 -07:00
torture.c torture: Wake up kthreads after storing task_struct pointer 2022-02-01 17:24:39 -08:00
tracepoint.c tracepoint: Fix kerneldoc comments 2021-08-16 11:39:51 -04:00
tsacct.c taskstats: version 12 with thread group and exe info 2022-04-29 14:38:03 -07:00
ucount.c ucounts: Handle wrapping in is_ucounts_overlimit 2022-02-17 09:11:57 -06:00
uid16.c
uid16.h
umh.c kthread: Don't allocate kthread_struct for init and umh 2022-05-06 14:49:44 -05:00
up.c
user_namespace.c ucounts: Fix systemd LimitNPROC with private users regression 2022-02-25 10:40:14 -06:00
user-return-notifier.c
user.c fs/epoll: use a per-cpu counter for user's watches count 2021-09-08 11:50:27 -07:00
usermode_driver.c blob_to_mnt(): kern_unmount() is needed to undo kern_mount() 2022-05-19 23:25:47 -04:00
utsname_sysctl.c
utsname.c
watch_queue.c watch_queue: Free the page array when watch_queue is dismantled 2022-04-02 10:37:39 -07:00
watchdog_hld.c printk: add functions to prefer direct printing 2022-04-22 21:30:58 +02:00
watchdog.c Not a lot of material this cycle. Many singleton patches against various 2022-05-27 11:22:03 -07:00
workqueue_internal.h workqueue: Assign a color to barrier work items 2021-08-17 07:49:10 -10:00
workqueue.c workqueue: Restrict kworker in the offline CPU pool running on housekeeping CPUs 2022-04-21 12:31:04 -10:00