linux-next/kernel
Kuan-Wei Chiu 92a8b224b8 lib/min_heap: introduce non-inline versions of min heap API functions
Patch series "Enhance min heap API with non-inline functions and
optimizations", v2.

Add non-inline versions of the min heap API functions in lib/min_heap.c
and updates all users outside of kernel/events/core.c to use these
non-inline versions.  To mitigate the performance impact of indirect
function calls caused by the non-inline versions of the swap and compare
functions, a builtin swap has been introduced that swaps elements based on
their size.  Additionally, it micro-optimizes the efficiency of the min
heap by pre-scaling the counter, following the same approach as in
lib/sort.c.  Documentation for the min heap API has also been added to the
core-api section.


This patch (of 10):

All current min heap API functions are marked with '__always_inline'. 
However, as the number of users increases, inlining these functions
everywhere leads to a increase in kernel size.

In performance-critical paths, such as when perf events are enabled and
min heap functions are called on every context switch, it is important to
retain the inline versions for optimal performance.  To balance this, the
original inline functions are kept, and additional non-inline versions of
the functions have been added in lib/min_heap.c.

Link: https://lkml.kernel.org/r/20241020040200.939973-1-visitorckw@gmail.com
Link: https://lore.kernel.org/20240522161048.8d8bbc7b153b4ecd92c50666@linux-foundation.org
Link: https://lkml.kernel.org/r/20241020040200.939973-2-visitorckw@gmail.com
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Ching-Chun (Jim) Huang <jserv@ccns.ncku.edu.tw>
Cc: Coly Li <colyli@suse.de>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Kuan-Wei Chiu <visitorckw@gmail.com>
Cc: "Liang, Kan" <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Matthew Sakai <msakai@redhat.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-11-05 17:12:34 -08:00
..
bpf BPF fixes: 2024-10-31 14:56:19 -10:00
cgroup cgroup: Fix potential overflow issue when checking max_depth 2024-10-14 13:39:25 -10:00
configs tinyconfig: remove unnecessary 'is not set' for choice blocks 2024-09-01 20:34:38 +09:00
debug move asm/unaligned.h to linux/unaligned.h 2024-10-02 17:23:23 -04:00
dma dma-mapping: report unlimited DMA addressing in IOMMU DMA path 2024-09-23 08:38:56 +02:00
entry treewide: context_tracking: Rename CONTEXT_* into CT_STATE_* 2024-07-29 07:33:10 +05:30
events lib/min_heap: introduce non-inline versions of min heap API functions 2024-11-05 17:12:34 -08:00
futex fault-inject: improve build for CONFIG_FAULT_INJECTION=n 2024-09-01 20:43:33 -07:00
gcov gcov: add support for GCC 14 2024-06-15 10:43:06 -07:00
irq genirq/msi: Fix off-by-one error in msi_domain_alloc() 2024-10-27 10:40:47 +01:00
kcsan kcsan: Use min() to fix Coccinelle warning 2024-08-01 16:40:44 -07:00
livepatch livepatch: Replace snprintf() with sysfs_emit() 2024-07-02 16:56:18 +02:00
locking Locking changes for v6.12: 2024-09-29 08:51:30 -07:00
module Modules changes for v6.12-rc1 2024-09-28 09:06:15 -07:00
power [tree-wide] finally take no_llseek out 2024-09-27 08:18:43 -07:00
printk drm next for 6.12-rc1 2024-09-19 10:18:15 +02:00
rcu Merge branch 'linus' into sched/urgent, to resolve conflict 2024-10-17 09:58:07 +02:00
sched A few scheduler fixes: 2024-11-03 08:18:28 -10:00
time Including fixes from netfiler, xfrm and bluetooth. 2024-10-24 16:43:50 -07:00
trace Fixes for function graph: 2024-10-27 08:56:22 -10:00
.gitignore
acct.c kernel misc: Remove the now superfluous sentinel elements from ctl_table array 2024-04-24 09:43:53 +02:00
async.c async: Use a dedicated unbound workqueue with raised min_active 2024-02-09 11:13:59 -10:00
audit_fsnotify.c
audit_tree.c fsnotify: create a wrapper fsnotify_find_inode_mark() 2024-04-04 16:24:16 +02:00
audit_watch.c fsnotify: create a wrapper fsnotify_find_inode_mark() 2024-04-04 16:24:16 +02:00
audit.c audit: Make use of str_enabled_disabled() helper 2024-09-03 16:35:16 -04:00
audit.h
auditfilter.c audit: use task_tgid_nr() instead of task_pid_nr() 2024-08-28 16:48:28 -04:00
auditsc.c auditsc: replace memcpy() with strscpy() 2024-11-05 17:12:29 -08:00
backtracetest.c backtracetest: add MODULE_DESCRIPTION() 2024-06-24 22:24:55 -07:00
bounds.c bounds: Use the right number of bits for power-of-two CONFIG_NR_CPUS 2024-04-29 08:29:29 -07:00
capability.c
cfi.c
compat.c
configs.c
context_tracking.c context_tracking, rcu: Rename rcu_dyntick trace event into rcu_watching 2024-08-15 21:30:43 +05:30
cpu_pm.c
cpu.c Updates for timers and timekeeping: 2024-09-17 07:25:37 +02:00
crash_core.c kexec/crash: no crash update when kexec in progress 2024-11-05 17:12:27 -08:00
crash_reserve.c crash: fix crash memory reserve exceed system memory bug 2024-09-01 20:43:30 -07:00
cred.c cred: Use KMEM_CACHE() instead of kmem_cache_create() 2024-02-23 17:33:31 -05:00
delayacct.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
dma.c
elfcorehdr.c crash: remove dependency of FA_DUMP on CRASH_DUMP 2024-02-23 17:48:22 -08:00
exec_domain.c
exit.c ALong with the usual shower of singleton patches, notable patch series in 2024-09-21 07:29:05 -07:00
exit.h exit: add internal include file with helpers 2023-09-21 12:03:50 -06:00
extable.c
fail_function.c
fork.c A single fix for posix CPU timers 2024-11-03 08:22:21 -10:00
freezer.c sched/fair: Fix external p->on_rq users 2024-10-14 09:14:35 +02:00
gen_kheaders.sh kheaders: use command -v to test for existence of cpio 2024-05-30 01:13:20 +09:00
groups.c groups: Convert group_info.usage to refcount_t 2023-09-29 11:28:39 -07:00
hung_task.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
iomem.c
irq_work.c
jump_label.c jump_label: Fix static_key_slow_dec() yet again 2024-09-10 11:57:27 +02:00
kallsyms_internal.h kallsyms: get rid of code for absolute kallsyms 2024-07-20 16:33:21 +09:00
kallsyms_selftest.c kallsyms: Match symbols exactly with CONFIG_LTO_CLANG 2024-08-15 09:33:35 -07:00
kallsyms_selftest.h
kallsyms.c kallsyms: Match symbols exactly with CONFIG_LTO_CLANG 2024-08-15 09:33:35 -07:00
kcmp.c file: convert to SLAB_TYPESAFE_BY_RCU 2023-10-19 11:02:48 +02:00
Kconfig.freezer
Kconfig.hz
Kconfig.kexec crash: clean up kdump related config items 2024-02-23 17:48:22 -08:00
Kconfig.locks
Kconfig.preempt sched_ext: Build fix on !CONFIG_STACKTRACE[_SUPPORT] 2024-08-01 07:08:01 -10:00
kcov.c Updates for KCOV instrumentation on x86: 2024-09-17 12:40:34 +02:00
kexec_core.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
kexec_elf.c
kexec_file.c kexec_file: fix elfcorehdr digest exclusion when CONFIG_CRASH_HOTPLUG=y 2024-09-01 17:59:01 -07:00
kexec_internal.h kexec: use atomic_try_cmpxchg_acquire() in kexec_trylock() 2024-09-01 20:43:23 -07:00
kexec.c crash: add a new kexec flag for hotplug support 2024-04-23 14:59:01 +10:00
kheaders.c
kprobes.c kprobes: Fix to check symbol prefixes correctly 2024-08-05 14:04:03 +09:00
ksyms_common.c
ksysfs.c profiling: remove prof_cpu_mask 2024-07-29 10:45:54 -07:00
kthread.c get rid of __get_task_comm() 2024-11-05 17:12:28 -08:00
latencytop.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
Makefile mm: move kernel/numa.c to mm/ 2024-09-03 21:15:26 -07:00
module_signature.c
notifier.c reboot: move reboot_notifier_list to kernel/reboot.c 2024-11-05 17:12:31 -08:00
nsproxy.c introduce fd_file(), convert all accessors to it. 2024-08-12 22:00:43 -04:00
padata.c This update includes the following changes: 2024-09-16 06:28:28 +02:00
panic.c drm next for 6.12-rc1 2024-09-19 10:18:15 +02:00
params.c params: Fix multi-line comment style 2023-12-01 09:51:44 -08:00
pid_namespace.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
pid_sysctl.h sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
pid.c introduce fd_file(), convert all accessors to it. 2024-08-12 22:00:43 -04:00
profile.c profiling: remove profile=sleep support 2024-08-04 13:36:28 -07:00
ptrace.c ptrace_attach: shift send(SIGSTOP) into ptrace_set_stopped() 2024-02-22 15:38:52 -08:00
range.c
reboot.c reboot: move reboot_notifier_list to kernel/reboot.c 2024-11-05 17:12:31 -08:00
regset.c regset: use kvzalloc() for regset_get_alloc() 2024-04-25 21:07:03 -07:00
relay.c [tree-wide] finally take no_llseek out 2024-09-27 08:18:43 -07:00
resource_kunit.c resource, kunit: fix user-after-free in resource_test_region_intersects() 2024-10-09 12:47:19 -07:00
resource.c resource: correct reallocate_resource() documentation 2024-11-05 17:12:31 -08:00
rseq.c
scftorture.c scftorture: Make torture_type static 2024-05-30 15:31:51 -07:00
scs.c
seccomp.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
signal.c Revert "binfmt_elf, coredump: Log the reason of the failed core dumps" 2024-09-26 11:39:02 -07:00
smp.c smp: print only local CPU info when sched_clock goes backward 2024-08-15 00:06:48 +05:30
smpboot.c kthread: add kthread_stop_put 2023-10-04 10:41:57 -07:00
smpboot.h
softirq.c softirq: Remove unused 'action' parameter from action callback 2024-08-20 17:13:40 +02:00
stackleak.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
stacktrace.c stacktrace: fix kernel-doc typo 2023-12-29 12:22:29 -08:00
static_call_inline.c static_call: Replace pointless WARN_ON() in static_call_module_notify() 2024-09-06 16:29:22 +02:00
static_call.c
stop_machine.c rcu: Rename rcu_momentary_dyntick_idle() into rcu_momentary_eqs() 2024-08-15 21:30:42 +05:30
sys_ni.c Probes updates for v6.11: 2024-07-18 12:19:20 -07:00
sys.c struct fd layout change (and conversion to accessor helpers) 2024-09-23 09:35:36 -07:00
sysctl-test.c sysctl: Add module description to sysctl-testing 2024-06-03 15:20:37 +02:00
sysctl.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
task_work.c sched/core: Disable page allocation in task_tick_mm_cid() 2024-10-11 10:49:32 +02:00
taskstats.c introduce fd_file(), convert all accessors to it. 2024-08-12 22:00:43 -04:00
torture.c torture: Add MODULE_DESCRIPTION() 2024-05-30 15:31:38 -07:00
tracepoint.c tracepoint: Support iterating tracepoints in a loading module 2024-09-25 23:23:44 +09:00
tsacct.c tsacct: replace strncpy() with strscpy() 2024-07-12 16:39:53 -07:00
ucount.c sysctl changes for v6.10-rc1 2024-05-17 17:31:24 -07:00
uid16.c
uid16.h
umh.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
up.c
user_namespace.c user_namespace: use kmemdup_array() instead of kmemdup() for multiple allocation 2024-09-09 16:47:42 -07:00
user-return-notifier.c
user.c uidgid: make sure we fit into one cacheline 2024-09-12 12:16:09 +02:00
usermode_driver.c
utsname_sysctl.c sysctl: treewide: constify the ctl_table argument of proc_handlers 2024-07-24 20:59:29 +02:00
utsname.c
vhost_task.c vhost_task: Handle SIGKILL by flushing work and exiting 2024-05-22 08:31:15 -04:00
vmcore_info.c mm: support only one page_type per page 2024-09-03 21:15:43 -07:00
watch_queue.c introduce fd_file(), convert all accessors to it. 2024-08-12 22:00:43 -04:00
watchdog_buddy.c
watchdog_perf.c watchdog/perf: properly initialize the turbo mode timestamp and rearm counter 2024-07-17 21:11:34 -07:00
watchdog.c kernel/watchdog: always restore watchdog_softlockup(,hardlockup)_user_enabled after proc show 2024-11-05 17:12:27 -08:00
workqueue_internal.h
workqueue.c workqueue: Changes for v6.12 2024-09-18 06:59:44 +02:00