Mateusz Guzik 51d821654b percpu_counter: add a cmpxchg-based _add_batch variant
Interrupt disable/enable trips are quite expensive on x86-64 compared to a
mere cmpxchg (note: no lock prefix!) and percpu counters are used quite
often.

With this change I get a bump of 1% ops/s for negative path lookups,
plugged into will-it-scale:

void testcase(unsigned long long *iterations, unsigned long nr)
{
        while (1) {
                int fd = open("/tmp/nonexistent", O_RDONLY);
                assert(fd == -1);

                (*iterations)++;
        }
}

The win would be higher if it was not for other slowdowns, but one has
to start somewhere.

Link: https://lkml.kernel.org/r/20240528204257.434817-1-mjguzik@gmail.com
Signed-off-by: Mateusz Guzik <mjguzik@gmail.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Dennis Zhou <dennis@kernel.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-06-24 22:25:03 -07:00
..
2024-01-19 11:59:11 -08:00
2024-02-01 13:06:40 +01:00
2018-08-16 12:14:42 -07:00
2023-11-03 07:08:36 -10:00
2021-01-21 14:06:00 -07:00
2022-03-07 12:48:35 -07:00
2021-08-19 09:02:55 +09:00
2024-04-25 21:07:06 -07:00
2023-02-02 22:50:01 -08:00
2023-02-02 22:50:01 -08:00
2021-01-03 20:05:18 -05:00
2024-05-21 15:29:01 -07:00
2022-03-07 12:48:35 -07:00
2022-04-29 14:38:01 -07:00
2022-10-03 14:03:21 -07:00
2024-03-11 09:38:17 -07:00
2021-08-19 09:02:55 +09:00
2024-04-25 21:07:05 -07:00
2024-05-22 11:53:02 -07:00
2023-08-21 13:46:25 -07:00
2023-04-17 18:01:23 +02:00
2021-07-08 11:48:20 -07:00
2023-10-16 12:44:06 -04:00
2023-08-24 16:20:18 -07:00
2023-10-16 12:44:06 -04:00
2018-10-16 13:45:44 +02:00
2022-10-03 17:34:32 -07:00
2021-07-08 11:48:20 -07:00
2024-02-15 12:17:28 -05:00
2022-06-03 10:34:34 -07:00
2024-02-20 20:47:32 -08:00
2021-06-18 11:43:09 +02:00
2024-05-21 15:29:01 -07:00
2024-04-16 17:22:18 +02:00
2022-01-20 08:52:54 +02:00
2024-05-19 16:12:38 -07:00
2024-05-02 17:48:09 -04:00