linux-next/mm
Kairui Song 20214ad102 mm, memcontrol: avoid duplicated memcg enable check
Patch series "mm/swap_cgroup: remove global swap cgroup lock", v3.

This series removes the global swap cgroup lock.  The critical section of
this lock is very short but it's still a bottle neck for mass parallel
swap workloads.

Up to 10% performance gain for tmpfs build kernel test on a 48c96t system
under memory pressure, and no regression for other cases:


This patch (of 3):

mem_cgroup_uncharge_swap() includes a mem_cgroup_disabled() check,
so the caller doesn't need to check that.

Link: https://lkml.kernel.org/r/20241218114633.85196-1-ryncsn@gmail.com
Link: https://lkml.kernel.org/r/20241218114633.85196-2-ryncsn@gmail.com
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Yosry Ahmed <yosryahmed@google.com>
Reviewed-by: Roman Gushchin <roman.gushchin@linux.dev>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Acked-by: Chris Li <chrisl@kernel.org>
Cc: Barry Song <baohua@kernel.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Yosry Ahmed <yosryahmed@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-12-18 19:51:08 -08:00
..
damon mm-damon-tests-vaddr-kunith-reduce-stack-consumption-fix 2024-12-18 19:50:51 -08:00
kasan mm:kasan: fix sparse warnings: Should it be static? 2024-12-18 19:50:37 -08:00
kfence mm/kfence: add a new kunit test test_use_after_free_read_nofault() 2024-11-14 22:49:19 -08:00
kmsan mm, kasan, kmsan: instrument copy_from/to_kernel_nofault 2024-11-06 20:11:14 -08:00
backing-dev.c writeback: support retrieving per group debug writeback stats of bdi 2024-05-05 17:53:51 -07:00
balloon_compaction.c mm: remove MIGRATE_SYNC_NO_COPY mode 2024-07-03 19:30:00 -07:00
bootmem_info.c bootmem: stop using page->index 2024-11-07 14:38:07 -08:00
cma_debug.c mm/cma_debug: show complete cma name in debugfs directories 2022-09-11 20:25:50 -07:00
cma_sysfs.c mm/cma: add sysfs file 'release_pages_success' 2024-02-22 10:24:57 -08:00
cma.c cma: enforce non-zero pageblock_order during cma_init_reserved_mem() 2024-11-14 22:49:19 -08:00
cma.h mm: change type of cma_area_count to unsigned int 2024-12-18 19:50:30 -08:00
compaction.c mm/page_alloc: move set_page_refcounted() to callers of post_alloc_hook() 2024-12-18 19:50:26 -08:00
debug_page_alloc.c mm: page_alloc: consolidate free page accounting 2024-04-25 20:56:04 -07:00
debug_page_ref.c
debug_vm_pgtable.c mm/debug_vm_pgtable: Use pxdp_get() for accessing page table entries 2024-09-17 01:07:01 -07:00
debug.c mm: open-code page_folio() in dump_page() 2024-12-05 19:54:45 -08:00
dmapool_test.c mm/dmapool: add MODULE_DESCRIPTION() 2024-07-03 19:29:58 -07:00
dmapool.c mm/mempool/dmapool: remove CONFIG_DEBUG_SLAB ifdefs 2023-12-05 11:17:58 +01:00
early_ioremap.c mm/early_ioremap: add null pointer checks to prevent NULL-pointer dereference 2024-12-18 19:51:03 -08:00
execmem.c alloc_tag: populate memory for module tags as needed 2024-11-07 14:25:16 -08:00
fadvise.c fdget(), trivial conversions 2024-11-03 01:28:06 -05:00
fail_page_alloc.c fault-inject: improve build for CONFIG_FAULT_INJECTION=n 2024-09-01 20:43:33 -07:00
failslab.c fault-inject: improve build for CONFIG_FAULT_INJECTION=n 2024-09-01 20:43:33 -07:00
filemap.c filemap: remove unused folio_add_wait_queue 2024-12-18 19:50:33 -08:00
folio-compat.c mm/writeback: add folio_mark_dirty_lock() 2024-11-05 11:14:32 +01:00
gup_test.c Merge mm-hotfixes-stable into mm-stable to pick up depended-upon changes. 2023-06-23 16:58:19 -07:00
gup_test.h mm/gup_test: start/stop/read functionality for PIN LONGTERM test 2022-11-08 17:37:15 -08:00
gup.c mm/hugetlb: support FOLL_FORCE|FOLL_WRITE 2024-12-18 19:50:52 -08:00
highmem.c mm/highmem: make nr_free_highpages() return "unsigned long" 2024-07-03 19:30:06 -07:00
hmm.c mm: provide mm_struct and address to huge_ptep_get() 2024-07-12 15:52:15 -07:00
huge_memory.c mm/huge_memory.c: rename shadowed local 2024-12-18 19:51:07 -08:00
hugetlb_cgroup.c mm/hugetlb_cgroup: avoid useless return in void function 2024-12-18 19:50:29 -08:00
hugetlb_vmemmap.c mm/hugetlb_vmemmap: don't synchronize_rcu() without HVO 2024-09-01 20:25:45 -07:00
hugetlb_vmemmap.h mm: hugetlb_vmemmap: fix reference to nonexistent file 2023-10-25 16:47:14 -07:00
hugetlb.c memcg/hugetlb: introduce mem_cgroup_charge_hugetlb 2024-12-18 19:51:02 -08:00
hwpoison-inject.c mm/hwpoison: add MODULE_DESCRIPTION() 2024-07-03 19:29:58 -07:00
init-mm.c mm: convert mm_lock_seq to a proper seqcount 2024-12-18 19:50:50 -08:00
internal.h mseal: remove can_do_mseal() 2024-12-18 19:50:52 -08:00
interval_tree.c
io-mapping.c
ioremap.c mm: ioremap: remove unneeded ioremap_allowed and iounmap_allowed 2023-08-18 10:12:36 -07:00
Kconfig mm: pgtable: reclaim empty PTE page in madvise(MADV_DONTNEED) 2024-12-18 19:50:47 -08:00
Kconfig.debug slub: Introduce CONFIG_SLUB_RCU_DEBUG 2024-08-27 14:12:51 +02:00
khugepaged.c mm: khugepaged: recheck pmd state in retract_page_tables() 2024-12-18 19:50:45 -08:00
kmemleak.c mm/kmemleak: fix sleeping function called from invalid context at print message 2024-12-18 19:49:57 -08:00
ksm.c - The series "zram: optimal post-processing target selection" from 2024-11-23 09:58:07 -08:00
list_lru.c mm/list_lru: simplify the list_lru walk callback function 2024-11-11 17:22:26 -08:00
maccess.c kasan: migrate copy_user_test to kunit 2024-11-11 00:26:44 -08:00
madvise.c mm: pgtable: reclaim empty PTE page in madvise(MADV_DONTNEED) 2024-12-18 19:50:47 -08:00
Makefile mm: pgtable: reclaim empty PTE page in madvise(MADV_DONTNEED) 2024-12-18 19:50:47 -08:00
mapping_dirty_helpers.c mm: fix clean_record_shared_mapping_range kernel-doc 2023-08-24 16:20:30 -07:00
memblock.c memblock: allow zero threshold in validate_numa_converage() 2024-12-01 21:08:56 +02:00
memcontrol-v1.c mm: remove the non-useful else after a break in a if statement 2024-12-18 19:50:35 -08:00
memcontrol-v1.h mm: memcg: declare do_memsw_account inline 2024-12-05 19:54:46 -08:00
memcontrol.c mm, memcontrol: avoid duplicated memcg enable check 2024-12-18 19:51:08 -08:00
memfd.c mm: perform all memfd seal checks in a single place 2024-12-18 19:50:51 -08:00
memory_hotplug.c mm/memory_hotplug: don't use __GFP_HARDWALL when migrating pages via memory offlining 2024-12-18 19:50:54 -08:00
memory-failure.c mm/memory-failure: replace sprintf() with sysfs_emit() 2024-11-11 00:26:46 -08:00
memory-tiers.c memory tiers: use default_dram_perf_ref_source in log message 2024-09-26 14:01:44 -07:00
memory.c mm: unexport apply_to_existing_page_range 2024-12-18 19:51:05 -08:00
mempolicy.c mm/mempolicy: add alloc_frozen_pages() 2024-12-18 19:50:28 -08:00
mempool.c mm: fix xyz_noprof functions calling profiled functions 2024-06-05 19:19:26 -07:00
memremap.c mm: convert put_devmap_managed_page_refs() to put_devmap_managed_folio_refs() 2024-05-05 17:53:49 -07:00
memtest.c memtest: use {READ,WRITE}_ONCE in memory scanning 2024-03-13 12:12:21 -07:00
migrate_device.c mm: remap unused subpages to shared zeropage when splitting isolated thp 2024-09-09 16:39:03 -07:00
migrate.c mm/migrate: skip migrating folios under writeback with AS_WRITEBACK_INDETERMINATE mappings 2024-12-18 19:51:06 -08:00
mincore.c mm: provide mm_struct and address to huge_ptep_get() 2024-07-12 15:52:15 -07:00
mlock.c mm/mlock: set the correct prev on failure 2024-11-07 14:14:58 -08:00
mm_init.c memblock: updates for 6.13-rc1 2024-11-27 11:13:25 -08:00
mm_slot.h mm: introduce common struct mm_slot 2022-10-03 14:02:43 -07:00
mmap_lock.c mm: mmap_lock: optimize mmap_lock tracepoints 2024-12-18 19:50:29 -08:00
mmap.c mm: add comments to do_mmap(), mmap_region() and vm_mmap() 2024-12-18 19:51:03 -08:00
mmu_gather.c x86: mm: free page table pages by RCU instead of semi RCU 2024-12-18 19:50:48 -08:00
mmu_notifier.c mm: move internal core VMA manipulation functions to own file 2024-09-01 20:25:54 -07:00
mmzone.c mm: improve code consistency with zonelist_* helper functions 2024-09-01 20:25:55 -07:00
mprotect.c mm: add PTE_MARKER_GUARD PTE marker 2024-11-11 00:26:44 -08:00
mremap.c - The series "zram: optimal post-processing target selection" from 2024-11-23 09:58:07 -08:00
mseal.c mseal: remove can_do_mseal() 2024-12-18 19:50:52 -08:00
msync.c mm/msync: use vma_find() instead of vma linked list 2022-09-26 19:46:25 -07:00
nommu.c nommu: pass NULL argument to vma_iter_prealloc() 2024-11-11 17:20:23 -08:00
numa_emulation.c mm: introduce numa_emulation 2024-09-03 21:15:31 -07:00
numa_memblks.c mm: numa_clear_kernel_node_hotplug: Add NUMA_NO_NODE check for node id 2024-10-28 21:40:40 -07:00
numa.c mm: make range-to-target_node lookup facility a part of numa_memblks 2024-09-03 21:15:32 -07:00
oom_kill.c mm: move mm flags to mm_types.h 2024-11-05 16:56:26 -08:00
page_alloc.c mm/page_alloc: don't use __GFP_HARDWALL when migrating pages via alloc_contig*() 2024-12-18 19:50:53 -08:00
page_counter.c mm, memcg: cg2 memory{.swap,}.peak write handlers 2024-09-01 20:25:53 -07:00
page_ext.c mm: don't account memmap per-node 2024-08-15 22:16:14 -07:00
page_frag_cache.c mm/page_alloc: export free_frozen_pages() instead of free_unref_page() 2024-12-18 19:50:25 -08:00
page_idle.c mm/page_idle: constify 'struct bin_attribute' 2024-12-18 19:51:07 -08:00
page_io.c mm: add per-order mTHP swpin counters 2024-11-11 00:26:43 -08:00
page_isolation.c mm/page_isolation: don't pass gfp flags to start_isolate_page_range() 2024-12-18 19:50:40 -08:00
page_owner.c mm/page-owner: use gfp_nested_mask() instead of open coded masking 2024-05-19 14:40:44 -07:00
page_poison.c mm/page_poison: replace kmap_atomic() with kmap_local_page() 2023-12-10 16:51:50 -08:00
page_reporting.c mm, treewide: rename MAX_ORDER to MAX_PAGE_ORDER 2024-01-08 15:27:15 -08:00
page_reporting.h
page_table_check.c mm/page_table_check: fix crash on ZONE_DEVICE 2024-06-15 10:43:04 -07:00
page_vma_mapped.c mm: mass constification of folio/page pointers 2024-11-07 14:38:07 -08:00
page-writeback.c mm/page-writeback: consolidate wb_thresh bumping logic into __wb_calc_thresh 2024-12-18 19:50:30 -08:00
pagewalk.c mm: pagewalk: add the ability to install PTEs 2024-11-11 00:26:44 -08:00
percpu-internal.h mm: remove CONFIG_MEMCG_KMEM 2024-07-10 12:14:54 -07:00
percpu-km.c
percpu-stats.c mm: use vmalloc_array and vcalloc for array allocations 2022-03-08 09:30:46 -05:00
percpu-vm.c percpu: clean up all mappings when pcpu_map_pages() fails 2024-04-25 20:55:49 -07:00
percpu.c mm: use page->private instead of page->index in percpu 2024-11-07 14:38:07 -08:00
pgalloc-track.h
pgtable-generic.c mm: add RCU annotation to pte_offset_map(_lock) 2024-12-18 19:04:43 -08:00
process_vm_access.c mm: refactor mm_access() to not return NULL 2024-11-05 16:56:23 -08:00
pt_reclaim.c mm-pgtable-reclaim-empty-pte-page-in-madvisemadv_dontneed-fix 2024-12-18 19:50:48 -08:00
ptdump.c mm: ptdump: add check_wx_pages debugfs attribute 2024-02-22 10:24:47 -08:00
readahead.c readahead: properly shorten readahead when falling back to do_page_cache_ra() 2024-12-18 19:50:41 -08:00
rmap.c mm: mass constification of folio/page pointers 2024-11-07 14:38:07 -08:00
rodata_test.c mm/rodata_test: verify test data is unchanged, rather than non-zero 2024-12-18 19:50:33 -08:00
secretmem.c secretmem: disable memfd_secret() if arch cannot set direct map 2024-10-09 12:47:19 -07:00
shmem_quota.c shmem_quota: build the object file conditionally to the config option 2024-09-01 20:25:45 -07:00
shmem.c mm: perform all memfd seal checks in a single place 2024-12-18 19:50:51 -08:00
show_mem.c mm/show_mem: use str_yes_no() helper in show_free_areas() 2024-11-07 14:38:08 -08:00
shrinker_debug.c mm: shrinker: use min() to improve shrinker_debugfs_scan_write() 2024-09-03 21:15:40 -07:00
shrinker.c mm: shrinker: avoid memleak in alloc_shrinker_info 2024-10-31 20:27:04 -07:00
shuffle.c mm/shuffle: convert module_param_call to module_param_cb 2022-10-03 14:03:07 -07:00
shuffle.h mm, treewide: rename MAX_ORDER to MAX_PAGE_ORDER 2024-01-08 15:27:15 -08:00
slab_common.c slab updates for 6.13 2024-11-25 16:51:24 -08:00
slab.h mm/slub: Avoid list corruption when removing a slab from the full list 2024-11-16 21:19:39 +01:00
slub.c mm/migrate: remove slab checks in isolate_movable_page() 2024-12-18 19:51:01 -08:00
sparse-vmemmap.c mm: define general function pXd_init() 2024-11-11 17:22:27 -08:00
sparse.c bootmem: stop using page->index 2024-11-07 14:38:07 -08:00
swap_cgroup.c mm-swap_cgroup-allocate-swap_cgroup-map-using-vcalloc-fix 2024-12-18 19:50:36 -08:00
swap_slots.c mm: swap: update get_swap_pages() to take folio order 2024-04-25 20:56:37 -07:00
swap_state.c mm: swap: use str_true_false() helper function 2024-11-06 20:11:14 -08:00
swap.c mm/mglru: rework workingset protection 2024-12-18 19:50:59 -08:00
swap.h mm: fix swap_read_folio_zeromap() for large folios with partial zeromap 2024-09-17 01:07:01 -07:00
swapfile.c mm, swap: fix allocation and scanning race with swapoff 2024-11-14 15:25:07 -08:00
truncate.c - The series "zram: optimal post-processing target selection" from 2024-11-23 09:58:07 -08:00
usercopy.c mm: Fix copy_from_user_nofault(). 2023-04-12 17:36:23 -07:00
userfaultfd.c mm-userfaultfd-recheck-dst_pmd-entry-in-move_pages_pte-fix 2024-12-18 19:50:46 -08:00
util.c mm: add comments to do_mmap(), mmap_region() and vm_mmap() 2024-12-18 19:51:03 -08:00
vma_internal.h mm/vma: add missing personality header import 2024-12-18 19:50:38 -08:00
vma.c mm: enforce __must_check on VMA merge and split 2024-12-18 19:50:51 -08:00
vma.h mm: enforce __must_check on VMA merge and split 2024-12-18 19:50:51 -08:00
vmalloc.c vmalloc: fix accounting with i915 2024-12-18 19:04:45 -08:00
vmpressure.c eventfd: simplify eventfd_signal() 2023-11-28 14:08:38 +01:00
vmscan.c mm: skip reclaiming folios in legacy memcg writeback indeterminate contexts 2024-12-18 19:51:06 -08:00
vmstat.c memcg/hugetlb: add hugeTLB counters to memcg 2024-11-14 22:49:19 -08:00
workingset.c mm/mglru: rework workingset protection 2024-12-18 19:50:59 -08:00
z3fold.c mm/z3fold: add __percpu annotation to *unbuddied pointer in struct z3fold_pool 2024-09-01 20:25:56 -07:00
zbud.c mm: zpool: return pool size in pages 2024-04-25 20:55:48 -07:00
zpool.c mm: zpool: return pool size in pages 2024-04-25 20:55:48 -07:00
zsmalloc.c mm/zsmalloc: use memcpy_from/to_page whereever possible 2024-11-07 14:38:07 -08:00
zswap.c mm/zswap: add LRU_STOP to comment about dropping the lru lock 2024-12-18 19:50:24 -08:00