Johannes Weiner ce00a96737 mm: memcontrol: revert use of root_mem_cgroup res_counter ...

Dave Hansen reports a massive scalability regression in an uncontained
page fault benchmark with more than 30 concurrent threads, which he
bisected down to 05b843012335 ("mm: memcontrol: use root_mem_cgroup
res_counter") and pin-pointed on res_counter spinlock contention.

That change relied on the per-cpu charge caches to mostly swallow the
res_counter costs, but it's apparent that the caches don't scale yet.

Revert memcg back to bypassing res_counters on the root level in order
to restore performance for uncontained workloads.

Reported-by: Dave Hansen <dave@sr71.net>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Dave Hansen <dave.hansen@intel.com>
Acked-by: Michal Hocko <mhocko@suse.cz>
Acked-by: Vladimir Davydov <vdavydov@parallels.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2014-09-05 08:19:02 -07:00

..

backing-dev.c

arch: Mass conversion of smp_mb__*()

2014-04-18 14:20:48 +02:00

balloon_compaction.c

mm: print more details for bad_page()

2014-01-23 16:36:50 -08:00

bootmem.c

mm/bootmem.c: remove unused local `map'

2013-11-13 12:09:09 +09:00

cleancache.c

mm: dump page when hitting a VM_BUG_ON using VM_BUG_ON_PAGE

2014-01-23 16:36:50 -08:00

cma.c

mm, CMA: clean-up log message

2014-08-06 18:01:16 -07:00

compaction.c

mm, compaction: properly signal and act upon lock and need_sched() contention

2014-06-04 16:54:11 -07:00

debug-pagealloc.c

mm, x86: Remove debug_pagealloc_enabled

2011-12-06 09:24:07 +01:00

dmapool.c

mm/dmapool.c: reuse devres_release() to free resources

2014-06-04 16:54:08 -07:00

early_ioremap.c

mm: create generic early_ioremap() support

2014-04-07 16:36:15 -07:00

fadvise.c

teach SYSCALL_DEFINE<n> how to deal with long long/unsigned long long

2013-03-03 22:46:22 -05:00

failslab.c

switch debugfs to umode_t

2012-01-03 22:54:56 -05:00

filemap_xip.c

seqcount: Add lockdep functionality to seqcount/seqlock structures

2013-11-06 12:40:26 +01:00

filemap.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2014-08-11 11:44:11 -07:00

fremap.c

mm: mark remap_file_pages() syscall as deprecated

2014-06-06 16:08:17 -07:00

frontswap.c

swap: change swap_list_head to plist, add swap_avail_head

2014-06-04 16:54:07 -07:00

gup.c

mm: describe mmap_sem rules for __lock_page_or_retry() and callers

2014-08-06 18:01:20 -07:00

highmem.c

mm/highmem: make kmap cache coloring aware

2014-08-06 18:01:22 -07:00

huge_memory.c

mm: memcontrol: rewrite charge API

2014-08-08 15:57:17 -07:00

hugetlb_cgroup.c

hugetlb_cgroup: use lockdep_assert_held rather than spin_is_locked

2014-08-29 16:28:16 -07:00

hugetlb.c

mm: fix potential infinite loop in dissolve_free_huge_pages()

2014-08-06 18:01:21 -07:00

hwpoison-inject.c

mm/hwpoison-inject.c: remove unnecessary null test before debugfs_remove_recursive

2014-08-06 18:01:19 -07:00

init-mm.c

atomic: use <linux/atomic.h>

2011-07-26 16:49:47 -07:00

internal.h

mm/internal.h: use nth_page

2014-08-06 18:01:16 -07:00

interval_tree.c

mm: add CONFIG_DEBUG_VM_RB build option

2012-10-09 16:22:42 +09:00

iov_iter.c

switch iov_iter_get_pages() to passing maximal number of pages

2014-08-07 14:40:11 -04:00

Kconfig

mm/zpool: update zswap to use zpool

2014-08-06 18:01:23 -07:00

Kconfig.debug

mm: more intensive memory corruption debugging

2012-01-10 16:30:42 -08:00

kmemcheck.c

kmemcheck: Fix build errors due to missing slab.h

2010-03-30 22:02:32 +09:00

kmemleak-test.c

mm/kmemleak-test.c: use pr_fmt for logging

2014-06-06 16:08:18 -07:00

kmemleak.c

mm: introduce kmemleak_update_trace()

2014-06-06 16:08:17 -07:00

ksm.c

sched: Remove proliferation of wait_on_bit() action functions

2014-07-16 15:10:39 +02:00

list_lru.c

mm: keep page cache radix tree nodes in check

2014-04-03 16:21:01 -07:00

maccess.c

mm: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

madvise.c

mm: update the description for madvise_remove

2014-08-06 18:01:18 -07:00

Makefile

mm/zpool: implement common zpool api to zbud/zsmalloc

2014-08-06 18:01:23 -07:00

memblock.c

memblock, memhotplug: fix wrong type in memblock_find_in_range_node().

2014-08-29 16:28:15 -07:00

memcontrol.c

mm: memcontrol: revert use of root_mem_cgroup res_counter

2014-09-05 08:19:02 -07:00

memory_hotplug.c

memory-hotplug: add zone_for_memory() for selecting zone for new memory

2014-08-06 18:01:21 -07:00

memory-failure.c

hwpoison: fix race with changing page during offlining

2014-08-06 18:01:19 -07:00

memory.c

x86,mm: fix pte_special versus pte_numa

2014-08-29 16:28:16 -07:00

mempolicy.c

Merge branch 'for-3.16-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup

2014-07-10 11:38:23 -07:00

mempool.c

mm/mempool.c: update the kmemleak stack trace for mempool allocations

2014-06-06 16:08:17 -07:00

migrate.c

mm: memcontrol: rewrite uncharge API

2014-08-08 15:57:17 -07:00

mincore.c

mm + fs: prepare for non-page entries in page cache radix trees

2014-04-03 16:21:00 -07:00

mlock.c

mm: describe mmap_sem rules for __lock_page_or_retry() and callers

2014-08-06 18:01:20 -07:00

mm_init.c

mm: bring back /sys/kernel/mm

2014-01-27 21:02:39 -08:00

mmap.c

mm: allow drivers to prevent new writable mappings

2014-08-08 15:57:31 -07:00

mmu_context.c

sched/mm: call finish_arch_post_lock_switch in idle_task_exit and use_mm

2014-02-21 08:50:17 +01:00

mmu_notifier.c

mmu_notifier: add call_srcu and sync function for listener to delay call and sync

2014-08-06 18:01:22 -07:00

mmzone.c

mm: numa: Change page last {nid,pid} into {cpu,pid}

2013-10-09 14:47:45 +02:00

mprotect.c

mm: move mmu notifier call from change_protection to change_pmd_range

2014-04-07 16:35:50 -07:00

mremap.c

mm, thp: close race between mremap() and split_huge_page()

2014-05-11 17:55:48 +09:00

msync.c

msync: fix incorrect fstart calculation

2014-07-03 09:21:53 -07:00

nobootmem.c

mm/memblock.c: call kmemleak directly from memblock_(alloc|free)

2014-06-06 16:08:17 -07:00

nommu.c

arm64,ia64,ppc,s390,sh,tile,um,x86,mm: remove default gate area

2014-08-08 15:57:27 -07:00

oom_kill.c

mm, oom: remove unnecessary exit_state check

2014-08-06 18:01:21 -07:00

page_alloc.c

mm, thp: restructure thp avoidance of light synchronous migration

2014-08-06 18:01:21 -07:00

page_cgroup.c

mm/page_cgroup.c: mark functions as static

2014-04-03 16:21:02 -07:00

page_io.c

fix __swap_writepage() compile failure on old gcc versions

2014-06-14 19:30:48 -05:00

page_isolation.c

mm: memory-hotplug: enable memory hotplug to handle hugepage

2013-09-11 15:57:48 -07:00

page-writeback.c

mm, writeback: prevent race when calculating dirty limits

2014-08-06 18:01:21 -07:00

pagewalk.c

mm/pagewalk.c: fix walk_page_range() access of wrong PTEs

2013-10-30 14:27:03 -07:00

percpu-km.c

percpu: clear memory allocated with the km allocator

2010-10-02 10:28:42 +03:00

percpu-vm.c

mm: fix kernel-doc warnings

2012-06-20 14:39:36 -07:00

percpu.c

percpu: Use ALIGN macro instead of hand coding alignment calculation

2014-06-19 11:00:27 -04:00

pgtable-generic.c

mm: actually clear pmd_numa before invalidating

2014-08-29 16:28:15 -07:00

process_vm_access.c

start adding the tag to iov_iter

2014-05-06 17:32:49 -04:00

quicklist.c

mm: delete various needless include <linux/module.h>

2011-10-31 09:20:11 -04:00

readahead.c

mm/readahead.c: remove unused file_ra_state from count_history_pages

2014-08-06 18:01:15 -07:00

rmap.c

mm: memcontrol: rewrite uncharge API

2014-08-08 15:57:17 -07:00

shmem.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2014-08-11 11:44:11 -07:00

slab_common.c

mm: move slab related stuff from util.c to slab_common.c

2014-08-06 18:01:15 -07:00

slab.c

Revert "slab: remove BAD_ALIEN_MAGIC"

2014-08-08 15:57:17 -07:00

slab.h

slab: convert last use of __FUNCTION__ to __func__

2014-08-06 18:01:15 -07:00

slob.c

slab: get_online_mems for kmem_cache_{create,destroy,shrink}

2014-06-04 16:53:59 -07:00

slub.c

slub: remove kmemcg id from create_unique_id

2014-08-06 18:01:21 -07:00

sparse-vmemmap.c

mm/sparse: use memblock apis for early memory allocations

2014-01-21 16:19:47 -08:00

sparse.c

mm: use macros from compiler.h instead of __attribute__((...))

2014-04-07 16:35:54 -07:00

swap_state.c

mm: allow drivers to prevent new writable mappings

2014-08-08 15:57:31 -07:00

swap.c

mm: memcontrol: use page lists for uncharge batching

2014-08-08 15:57:18 -07:00

swapfile.c

mm: memcontrol: rewrite uncharge API

2014-08-08 15:57:17 -07:00

truncate.c

mm: memcontrol: rewrite uncharge API

2014-08-08 15:57:17 -07:00

util.c

vm_is_stack: use for_each_thread() rather then buggy while_each_thread()

2014-08-08 15:57:17 -07:00

vmacache.c

mm,vmacache: optimize overflow system-wide flushing

2014-06-04 16:53:57 -07:00

vmalloc.c

mm/vmalloc.c: clean up map_vm_area third argument

2014-08-06 18:01:19 -07:00

vmpressure.c

arm, pm, vmpressure: add missing slab.h includes

2014-02-03 13:24:01 -05:00

vmscan.c

mm: memcontrol: use page lists for uncharge batching

2014-08-08 15:57:18 -07:00

vmstat.c

mm: vmscan: only update per-cpu thresholds for online CPU

2014-08-06 18:01:20 -07:00

workingset.c

mm: keep page cache radix tree nodes in check

2014-04-03 16:21:01 -07:00

zbud.c

mm/zpool: use prefixed module loading

2014-08-29 16:28:16 -07:00

zpool.c

mm/zpool: use prefixed module loading

2014-08-29 16:28:16 -07:00

zsmalloc.c

mm/zpool: use prefixed module loading

2014-08-29 16:28:16 -07:00

zswap.c

mm/zswap.c: add __init to zswap_entry_cache_destroy()

2014-08-08 15:57:18 -07:00