Martin KaFai Lau bb9b9f8802 bpf: Only set node->ref = 1 if it has not been set ...

This patch writes 'node->ref = 1' only if node->ref is 0.
The number of lookups/s for a ~1M entries LRU map increased by
~30% (260097 to 343313).

Other writes on 'node->ref = 0' is not changed.  In those cases, the
same cache line has to be changed anyway.

First column: Size of the LRU hash
Second column: Number of lookups/s

Before:
> echo "$((2**20+1)): $(./map_perf_test 1024 1 $((2**20+1)) 10000000 | awk '{print $3}')"
1048577: 260097

After:
> echo "$((2**20+1)): $(./map_perf_test 1024 1 $((2**20+1)) 10000000 | awk '{print $3}')"
1048577: 343313

Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>

2017-09-01 09:57:39 -07:00

..

bpf

bpf: Only set node->ref = 1 if it has not been set

2017-09-01 09:57:39 -07:00

cgroup

cpuset: fix a deadlock due to incomplete patching of cpusets_enabled()

2017-08-02 17:16:12 -07:00

configs

config: android-base: disable CONFIG_NFSD and CONFIG_NFS_FS

2017-06-09 11:47:38 +02:00

debug

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>

2017-03-02 08:42:34 +01:00

events

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2017-08-21 17:06:42 -07:00

gcov

gcov: support GCC 7.1

2017-05-12 15:57:15 -07:00

irq

genirq/ipi: Fixup checks against nr_cpu_ids

2017-08-20 10:49:05 +02:00

livepatch

livepatch: Fix stacking of patches with respect to RCU

2017-06-20 10:42:19 +02:00

locking

Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2017-07-21 11:11:23 -07:00

power

mm: fix global NR_SLAB_.*CLAIMABLE counter reads

2017-08-10 15:54:06 -07:00

printk

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk

2017-07-05 11:11:26 -07:00

rcu

rcu: Remove RCU CPU stall warnings from Tiny RCU

2017-06-08 18:52:45 -07:00

sched

sched/core: Fix some documentation build warnings

2017-07-25 11:17:02 +02:00

time

timers: Fix overflow in get_next_timer_interrupt

2017-08-01 14:20:53 +02:00

trace

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2017-08-15 20:23:23 -07:00

.gitignore

certs: add .gitignore to stop git nagging about x509_certificate_list

2015-10-21 15:18:35 +01:00

acct.c

sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>

2017-03-02 08:42:39 +01:00

async.c

async: Adjust system_state checks

2017-05-23 10:01:37 +02:00

audit_fsnotify.c

Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2017-05-03 11:05:15 -07:00

audit_tree.c

Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs

2017-05-03 11:05:15 -07:00

audit_watch.c

audit/stable-4.13 PR 20170816

2017-08-16 16:48:34 -07:00

audit.c

Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit

2017-07-20 10:22:26 -07:00

audit.h

audit: style fix

2017-06-12 18:07:43 -04:00

auditfilter.c

audit: kernel generated netlink traffic should have a portid of 0

2017-05-02 10:16:05 -04:00

auditsc.c

Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit

2017-07-05 11:24:05 -07:00

backtracetest.c

…

bounds.c

…

capability.c

capability: export has_capability

2017-01-12 07:01:56 -07:00

compat.c

Merge branch 'misc.compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2017-07-06 20:57:13 -07:00

configs.c

Replace <asm/uaccess.h> with <linux/uaccess.h> globally

2016-12-24 11:46:01 -08:00

context_tracking.c

context_tracking: Switch to new static_branch API

2015-11-24 09:56:43 +01:00

cpu_pm.c

…

cpu.c

smp/hotplug: Replace BUG_ON and react useful

2017-07-11 22:25:44 +02:00

crash_core.c

kdump: protect vmcoreinfo data under the crash memory

2017-07-12 16:26:00 -07:00

crash_dump.c

…

cred.c

doc: ReSTify credentials.txt

2017-05-18 10:30:19 -06:00

delayacct.c

sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>

2017-03-02 08:42:39 +01:00

dma.c

…

elfcore.c

…

exec_domain.c

…

exit.c

kernel/exit.c: avoid undefined behaviour when calling wait4()

2017-07-10 16:32:36 -07:00

extable.c

lib/extable.c: use bsearch() library function in search_extable()

2017-07-10 16:32:35 -07:00

fork.c

mm: migrate: prevent racy access to tlb_flush_pending

2017-08-10 15:54:07 -07:00

freezer.c

freezer, oom: check TIF_MEMDIE on the correct task

2016-07-28 16:07:41 -07:00

futex_compat.c

Replace <asm/uaccess.h> with <linux/uaccess.h> globally

2016-12-24 11:46:01 -08:00

futex.c

futex: Remove unnecessary warning from get_futex_key

2017-08-09 14:00:54 -07:00

groups.c

kernel/groups.c: use sort library function

2017-07-10 16:32:34 -07:00

hung_task.c

kernel/hung_task.c: defer showing held locks

2017-05-08 17:15:10 -07:00

irq_work.c

treewide: Remove old email address

2015-11-23 09:44:58 +01:00

jump_label.c

jump_label: Reorder hotplug lock and jump_label_lock

2017-05-26 10:10:45 +02:00

kallsyms.c

kernel/kallsyms.c: replace all_var with IS_ENABLED(CONFIG_KALLSYMS_ALL)

2017-07-10 16:32:34 -07:00

kcmp.c

kcmp: add KCMP_EPOLL_TFD mode to compare epoll target files

2017-07-12 16:26:01 -07:00

Kconfig.freezer

…

Kconfig.hz

…

Kconfig.locks

locking/mutex: Allow MUTEX_SPIN_ON_OWNER when DEBUG_MUTEXES

2016-10-25 11:31:51 +02:00

Kconfig.preempt

…

kcov.c

kcov: simplify interrupt check

2017-05-08 17:15:12 -07:00

kexec_core.c

kdump: protect vmcoreinfo data under the crash memory

2017-07-12 16:26:00 -07:00

kexec_file.c

kexec_file: adjust declaration of kexec_purgatory

2017-07-12 16:26:02 -07:00

kexec_internal.h

kexec_file: adjust declaration of kexec_purgatory

2017-07-12 16:26:02 -07:00

kexec.c

kdump: protect vmcoreinfo data under the crash memory

2017-07-12 16:26:00 -07:00

kmod.c

kmod: fix wait on recursive loop

2017-08-18 15:32:01 -07:00

kprobes.c

kprobes: Ensure that jprobe probepoints are at function entry

2017-07-08 11:05:35 +02:00

ksysfs.c

kexec: move vmcoreinfo out of the kernel's .bss section

2017-07-12 16:25:59 -07:00

kthread.c

cgroup, kthread: close race window where new kthreads can be migrated to non-root cgroups

2017-03-17 10:18:47 -04:00

latencytop.c

sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>

2017-03-02 08:42:39 +01:00

Makefile

kernel/watchdog: split up config options

2017-07-12 16:26:02 -07:00

membarrier.c

Fix: Disable sys_membarrier when nohz_full is enabled

2017-01-23 11:32:16 -08:00

memremap.c

mm, memory_hotplug: replace for_device by want_memblock in arch_add_memory

2017-07-06 16:24:32 -07:00

module_signing.c

KEYS: Move the point of trust determination to __key_link()

2016-04-11 22:43:43 +01:00

module-internal.h

…

module.c

Modules updates for v4.13

2017-07-12 17:22:01 -07:00

notifier.c

kernel/notifier.c: simplify expression

2017-02-24 17:46:56 -08:00

nsproxy.c

perf: Add PERF_RECORD_NAMESPACES to include namespaces related info

2017-03-13 15:57:41 -03:00

padata.c

padata: Avoid nested calls to cpus_read_lock() in pcrypt_init_padata()

2017-05-26 10:10:37 +02:00

panic.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>

2017-03-02 08:42:34 +01:00

params.c

boot/param: Move next_arg() function to lib/cmdline.c for later reuse

2017-04-18 10:37:13 +02:00

pid_namespace.c

pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes

2017-05-13 17:26:01 -05:00

pid.c

pids: make task_tgid_nr_ns() safe

2017-08-21 12:47:31 -07:00

profile.c

sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>

2017-03-02 08:42:39 +01:00

ptrace.c

ptrace: Properly initialize ptracer_cred on fork

2017-05-23 07:40:44 -05:00

range.c

…

reboot.c

…

relay.c

Merge branch 'work.splice' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2017-05-02 11:38:06 -07:00

resource.c

/proc/iomem: only expose physical resource addresses to privileged users

2016-04-14 12:56:09 -07:00

seccomp.c

seccomp: Switch from atomic_t to recount_t

2017-06-26 09:24:00 -07:00

signal.c

signal: don't remove SIGNAL_UNKILLABLE for traced tasks.

2017-08-18 15:32:02 -07:00

smp.c

smp, cpumask: Use non-atomic cpumask_{set,clear}_cpu()

2017-05-23 10:01:32 +02:00

smpboot.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>

2017-03-02 08:42:35 +01:00

smpboot.h

cpu/hotplug: Create hotplug threads

2016-03-01 20:36:56 +01:00

softirq.c

sched/core: Remove 'task' parameter and rename tsk_restore_flags() to current_restore_flags()

2017-04-11 09:06:32 +02:00

stacktrace.c

stacktrace/x86: add function for detecting reliable stack traces

2017-03-08 09:18:02 +01:00

stop_machine.c

stop_machine: Provide stop_machine_cpuslocked()

2017-05-26 10:10:36 +02:00

sys_ni.c

move aio compat to fs/aio.c

2016-12-22 22:58:37 -05:00

sys.c

fix a braino in compat_sys_getrlimit()

2017-07-12 09:15:00 -07:00

sysctl_binary.c

kernel/sysctl_binary.c: check name array length in deprecated_sysctl_warning()

2017-07-12 16:26:00 -07:00

sysctl.c

kernel/watchdog: split up config options

2017-07-12 16:26:02 -07:00

task_work.c

task_work: use READ_ONCE/lockless_dereference, avoid pi_lock if !task_works

2016-08-02 19:35:02 -04:00

taskstats.c

taskstats: add e/u/stime for TGID command

2017-05-08 17:15:12 -07:00

test_kprobes.c

…

torture.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/clock.h>

2017-03-02 08:42:27 +01:00

tracepoint.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>

2017-03-02 08:42:35 +01:00

tsacct.c

sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>

2017-03-02 08:42:39 +01:00

ucount.c

ucount: Remove the atomicity from ucount->count

2017-03-06 15:26:37 -06:00

uid16.c

sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h>

2017-03-02 08:42:31 +01:00

up.c

smp: Add function to execute a function synchronously on a CPU

2016-09-05 13:52:39 +02:00

user_namespace.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h>

2017-03-02 08:42:29 +01:00

user-return-notifier.c

…

user.c

sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>

2017-03-02 08:42:29 +01:00

utsname_sysctl.c

sched/headers: Remove <linux/rwsem.h> from <linux/sched.h>

2017-03-03 01:45:36 +01:00

utsname.c

sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h>

2017-03-02 08:42:38 +01:00

watchdog_hld.c

kernel/watchdog: Prevent false positives with turbo modes

2017-08-18 12:35:02 +02:00

watchdog.c

kernel/watchdog: Prevent false positives with turbo modes

2017-08-18 12:35:02 +02:00

workqueue_internal.h

sched/core: Get rid of 'cpu' argument in wq_worker_sleeping()

2016-03-02 10:28:47 -05:00

workqueue.c

workqueue: Work around edge cases for calc of pool's cpumask

2017-07-28 11:05:52 -04:00