Yafang Shao ceeadb83ae sched: Make struct sched_statistics independent of fair sched class ... If we want to use the schedstats facility to trace other sched classes, we should make it independent of fair sched class. The struct sched_statistics is the schedular statistics of a task_struct or a task_group. So we can move it into struct task_struct and struct task_group to achieve the goal. After the patch, schestats are orgnized as follows, struct task_struct { ... struct sched_entity se; struct sched_rt_entity rt; struct sched_dl_entity dl; ... struct sched_statistics stats; ... }; Regarding the task group, schedstats is only supported for fair group sched, and a new struct sched_entity_stats is introduced, suggested by Peter - struct sched_entity_stats { struct sched_entity se; struct sched_statistics stats; } __no_randomize_layout; Then with the se in a task_group, we can easily get the stats. The sched_statistics members may be frequently modified when schedstats is enabled, in order to avoid impacting on random data which may in the same cacheline with them, the struct sched_statistics is defined as cacheline aligned. As this patch changes the core struct of scheduler, so I verified the performance it may impact on the scheduler with 'perf bench sched pipe', suggested by Mel. Below is the result, in which all the values are in usecs/op. Before After kernel.sched_schedstats=0 5.2~5.4 5.2~5.4 kernel.sched_schedstats=1 5.3~5.5 5.3~5.5 [These data is a little difference with the earlier version, that is because my old test machine is destroyed so I have to use a new different test machine.] Almost no impact on the sched performance. No functional change. [lkp@intel.com: reported build failure in earlier version] Signed-off-by: Yafang Shao <laoar.shao@gmail.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Mel Gorman <mgorman@suse.de> Link: https://lore.kernel.org/r/20210905143547.4668-3-laoar.shao@gmail.com		2021-10-05 15:51:45 +02:00
..
bpf	bpf: Exempt CAP_BPF from checks against bpf_jit_limit	2021-09-28 09:28:37 +02:00
cgroup	bpf, cgroup: Assign cgroup in cgroup_sk_alloc when called from interrupt	2021-09-28 09:29:19 +02:00
configs	drivers/char: remove /dev/kmem for good	2021-05-07 00:26:34 -07:00
debug	kgdb patches for 5.15	2021-09-07 12:08:04 -07:00
dma	dma-mapping fixes for Linux 5.15	2021-09-17 11:54:48 -07:00
entry	entry: rseq: Call rseq_handle_notify_resume() in tracehook_notify_resume()	2021-09-22 10:24:01 -04:00
events	perf/core: fix userpage->time_enabled of inactive events	2021-10-01 13:57:54 +02:00
gcov	Kconfig: Introduce ARCH_WANTS_NO_INSTR and CC_HAS_NO_PROFILE_FN_ATTR	2021-06-22 11:07:18 -07:00
irq	irqchip fixes for 5.15, take #1	2021-09-24 14:11:04 +02:00
kcsan	LKMM updates:	2021-09-02 13:00:15 -07:00
livepatch	livepatch: Replace deprecated CPU-hotplug functions.	2021-08-19 12:00:24 +02:00
locking	locking/rwbase: Take care of ordering guarantee for fastpath reader	2021-09-15 17:49:16 +02:00
power	Merge branches 'pm-pci', 'pm-sleep', 'pm-domains' and 'powercap'	2021-08-30 19:25:42 +02:00
printk	memblock: introduce saner 'memblock_free_ptr()' interface	2021-09-14 13:23:22 -07:00
rcu	Updates for locking and atomics:	2021-08-30 14:26:36 -07:00
sched	sched: Make struct sched_statistics independent of fair sched class	2021-10-05 15:51:45 +02:00
time	posix-cpu-timers: Prevent spuriously armed 0-value itimer	2021-09-23 11:53:51 +02:00
trace	blktrace: Fix uaf in blk_trace access after removing by sysfs	2021-09-24 11:06:15 -06:00
.gitignore	.gitignore: prefix local generated files with a slash	2021-05-02 00:43:35 +09:00
acct.c	kernel/acct.c: use dedicated helper to access rlimit values	2021-09-08 11:50:26 -07:00
async.c	kernel/async.c: remove async_unregister_domain()	2021-05-07 00:26:33 -07:00
audit_fsnotify.c	audit_alloc_mark(): don't open-code ERR_CAST()	2021-02-23 10:25:27 -05:00
audit_tree.c	audit: move put_tree() to avoid trim_trees refcount underflow and UAF	2021-08-24 18:52:36 -04:00
audit_watch.c	fsnotify: generalize handle_inode_event()	2020-12-03 14:58:35 +01:00
audit.c	lsm: separate security_task_getsecid() into subjective and objective variants	2021-03-22 15:23:32 -04:00
audit.h	audit: add header protection to kernel/audit.h	2021-07-19 22:38:24 -04:00
auditfilter.c	lsm: separate security_task_getsecid() into subjective and objective variants	2021-03-22 15:23:32 -04:00
auditsc.c	audit: remove trailing spaces and tabs	2021-06-10 20:59:05 -04:00
backtracetest.c
bounds.c
capability.c	capability: handle idmapped mounts	2021-01-24 14:27:16 +01:00
cfi.c	cfi: Use rcu_read_{un}lock_sched_notrace	2021-08-11 13:11:12 -07:00
compat.c	arch: remove compat_alloc_user_space	2021-09-08 15:32:35 -07:00
configs.c
context_tracking.c
cpu_pm.c	PM: cpu: Make notifier chain use a raw_spinlock_t	2021-08-16 18:55:32 +02:00
cpu.c	cpu/hotplug: Add debug printks for hotplug callback failures	2021-08-10 18:31:32 +02:00
crash_core.c	kdump: use vmlinux_build_id to simplify	2021-07-08 11:48:22 -07:00
crash_dump.c
cred.c	ucounts: Increase ucounts reference counter before the security hook	2021-08-23 16:13:04 -05:00
delayacct.c	delayacct: Add sysctl to enable at runtime	2021-05-12 11:43:25 +02:00
dma.c
exec_domain.c
exit.c	io_uring: remove files pointer in cancellation functions	2021-08-23 13:10:37 -06:00
extable.c
fail_function.c	fault-injection: handle EI_ETYPE_TRUE	2020-12-15 22:46:19 -08:00
fork.c	mm/hugetlb: initialize hugetlb_usage in mm_init	2021-09-08 18:45:53 -07:00
freezer.c	sched: Add get_current_state()	2021-06-18 11:43:08 +02:00
futex.c	futex: Remove unused variable 'vpid' in futex_proxy_trylock_atomic()	2021-09-03 23:00:22 +02:00
gen_kheaders.sh	kbuild: clean up ${quiet} checks in shell scripts	2021-05-27 04:01:50 +09:00
groups.c	groups: simplify struct group_info allocation	2021-02-26 09:41:03 -08:00
hung_task.c	Merge branch 'akpm' (patches from Andrew)	2021-07-02 12:08:10 -07:00
iomem.c
irq_work.c	irq_work: Make irq_work_queue() NMI-safe again	2021-06-10 10:00:08 +02:00
jump_label.c	jump_label: Fix jump_label_text_reserved() vs __init	2021-07-05 10:46:20 +02:00
kallsyms.c	module: add printk formats to add module build ID to stacktraces	2021-07-08 11:48:22 -07:00
kcmp.c	Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2020-12-15 19:36:48 -08:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks	locking/rwlock: Provide RT variant	2021-08-17 17:50:51 +02:00
Kconfig.preempt	sched/core: Disable CONFIG_SCHED_CORE by default	2021-06-28 22:43:05 +02:00
kcov.c	kernel: make kcov_common_handle consider the current context	2020-11-02 18:00:20 -08:00
kexec_core.c	Merge branch 'rework/printk_safe-removal' into for-linus	2021-08-30 16:36:10 +02:00
kexec_elf.c
kexec_file.c	kernel: kexec_file: fix error return code of kexec_calculate_store_digests()	2021-05-07 00:26:32 -07:00
kexec_internal.h	kexec: move machine_kexec_post_load() to public interface	2021-02-22 12:33:26 +00:00
kexec.c	kexec: avoid compat_alloc_user_space	2021-09-08 15:32:34 -07:00
kheaders.c
kmod.c	modules: add CONFIG_MODPROBE_PATH	2021-05-07 00:26:33 -07:00
kprobes.c	Locking fixes:	2021-07-11 11:06:09 -07:00
ksysfs.c
kthread.c	Merge branch 'akpm' (patches from Andrew)	2021-06-29 17:29:11 -07:00
latencytop.c
Makefile	kbuild: update config_data.gz only when the content of .config is changed	2021-05-02 00:43:35 +09:00
module_signature.c	module: harden ELF info handling	2021-01-19 10:24:45 +01:00
module_signing.c	module: harden ELF info handling	2021-01-19 10:24:45 +01:00
module-internal.h
module.c	printk: Userspace format indexing support	2021-07-19 11:57:48 +02:00
notifier.c	notifier: Remove atomic_notifier_call_chain_robust()	2021-08-16 18:55:32 +02:00
nsproxy.c	memcg: enable accounting for new namesapces and struct nsproxy	2021-09-03 09:58:12 -07:00
padata.c	padata: Remove repeated verbose license text	2021-08-27 16:30:18 +08:00
panic.c	Merge branch 'rework/printk_safe-removal' into for-linus	2021-08-30 16:36:10 +02:00
params.c	params: lift param_set_uint_minmax to common code	2021-08-16 14:42:22 +02:00
pid_namespace.c	memcg: enable accounting for new namesapces and struct nsproxy	2021-09-03 09:58:12 -07:00
pid.c	kernel/pid.c: implement additional checks upon pidfd_create() parameters	2021-08-10 12:53:07 +02:00
profile.c	profiling: fix shift-out-of-bounds bugs	2021-09-08 11:50:26 -07:00
ptrace.c	sched: Change task_struct::state	2021-06-18 11:43:09 +02:00
range.c	kernel.h: split out min()/max() et al. helpers	2020-10-16 11:11:19 -07:00
reboot.c	reboot: Add hardware protection power-off	2021-06-21 13:08:36 +01:00
regset.c
relay.c	relay: allow the use of const callback structs	2020-12-15 22:46:18 -08:00
resource_kunit.c	resource: provide meaningful MODULE_LICENSE() in test suite	2020-11-25 18:52:35 +01:00
resource.c	kernel/resource: fix return code check in __request_free_mem_region	2021-05-14 19:41:32 -07:00
rseq.c	KVM: rseq: Update rseq when processing NOTIFY_RESUME on xfer to KVM guest	2021-09-22 10:24:01 -04:00
scftorture.c	scftorture: Avoid NULL pointer exception on early exit	2021-07-27 11:39:30 -07:00
scs.c	scs: switch to vmapped shadow stacks	2020-12-01 10:30:28 +00:00
seccomp.c	Merge branch 'exit-cleanups-for-v5.15' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2021-09-01 14:52:05 -07:00
signal.c	Merge branch 'akpm' (patches from Andrew)	2021-09-03 10:08:28 -07:00
smp.c	smp: Fix all kernel-doc warnings	2021-08-11 14:47:16 +02:00
smpboot.c	smpboot: Replace deprecated CPU-hotplug functions.	2021-08-10 14:57:42 +02:00
smpboot.h
softirq.c	genirq: Change force_irqthreads to a static key	2021-08-10 22:50:07 +02:00
stackleak.c	stackleak: let stack_erasing_sysctl take a kernel pointer buffer	2020-09-19 13:13:39 -07:00
stacktrace.c	stacktrace: Remove reliable argument from arch_stack_walk() callback	2020-09-18 14:24:16 +01:00
static_call.c	static_call: Fix static_call_text_reserved() vs __init	2021-07-05 10:46:33 +02:00
stop_machine.c	stop_machine: Add caller debug info to queue_stop_cpus_work	2021-03-23 16:01:58 +01:00
sys_ni.c	compat: remove some compat entry points	2021-09-08 15:32:35 -07:00
sys.c	Merge branch 'akpm' (patches from Andrew)	2021-09-08 12:55:35 -07:00
sysctl-test.c	kernel/sysctl-test: Remove some casts which are no-longer required	2021-06-23 16:41:24 -06:00
sysctl.c	Merge branch 'akpm' (patches from Andrew)	2021-09-03 10:08:28 -07:00
task_work.c	kasan: record task_work_add() call stack	2021-04-30 11:20:42 -07:00
taskstats.c	treewide: rename nla_strlcpy to nla_strscpy.	2020-11-16 08:08:54 -08:00
test_kprobes.c
torture.c	torture: Replace deprecated CPU-hotplug functions.	2021-08-10 10:48:07 -07:00
tracepoint.c	tracepoint: Fix kerneldoc comments	2021-08-16 11:39:51 -04:00
tsacct.c
ucount.c	ucounts: add missing data type changes	2021-08-09 15:45:02 -05:00
uid16.c
uid16.h
umh.c	kernel/umh.c: fix some spelling mistakes	2021-05-07 00:26:34 -07:00
up.c	A set of locking related fixes and updates:	2021-05-09 13:07:03 -07:00
user_namespace.c	memcg: enable accounting for new namesapces and struct nsproxy	2021-09-03 09:58:12 -07:00
user-return-notifier.c
user.c	fs/epoll: use a per-cpu counter for user's watches count	2021-09-08 11:50:27 -07:00
usermode_driver.c	Merge branch 'work.namei' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2021-07-03 11:41:14 -07:00
utsname_sysctl.c
utsname.c	uts: Use generic ns_common::count	2020-08-19 14:13:20 +02:00
watch_queue.c	watch_queue: rectify kernel-doc for init_watch()	2021-01-26 11:16:34 +00:00
watchdog_hld.c
watchdog.c	kernel: watchdog: modify the explanation related to watchdog thread	2021-06-29 10:53:46 -07:00
workqueue_internal.h	workqueue: Assign a color to barrier work items	2021-08-17 07:49:10 -10:00
workqueue.c	workqueue: Assign a color to barrier work items	2021-08-17 07:49:10 -10:00