Thomas Gleixner 683be13a28 timer: Minimize nohz off overhead
If nohz is disabled on the kernel command line the [hr]timer code
still calls wake_up_nohz_cpu() and tick_nohz_full_cpu(), a pretty
pointless exercise. Cache nohz_active in [hr]timer per cpu bases and
avoid the overhead.

Before:
  48.10%  hog       [.] main
  15.25%  [kernel]  [k] _raw_spin_lock_irqsave
   9.76%  [kernel]  [k] _raw_spin_unlock_irqrestore
   6.50%  [kernel]  [k] mod_timer
   6.44%  [kernel]  [k] lock_timer_base.isra.38
   3.87%  [kernel]  [k] detach_if_pending
   3.80%  [kernel]  [k] del_timer
   2.67%  [kernel]  [k] internal_add_timer
   1.33%  [kernel]  [k] __internal_add_timer
   0.73%  [kernel]  [k] timerfn
   0.54%  [kernel]  [k] wake_up_nohz_cpu

After:
  48.73%  hog       [.] main
  15.36%  [kernel]  [k] _raw_spin_lock_irqsave
   9.77%  [kernel]  [k] _raw_spin_unlock_irqrestore
   6.61%  [kernel]  [k] lock_timer_base.isra.38
   6.42%  [kernel]  [k] mod_timer
   3.90%  [kernel]  [k] detach_if_pending
   3.76%  [kernel]  [k] del_timer
   2.41%  [kernel]  [k] internal_add_timer
   1.39%  [kernel]  [k] __internal_add_timer
   0.76%  [kernel]  [k] timerfn

We probably should have a cached value for nohz full in the per cpu
bases as well to avoid the cpumask check. The base cache line is hot
already, the cpumask not necessarily.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Joonwoo Park <joonwoop@codeaurora.org>
Cc: Wenbo Wang <wenbo.wang@memblaze.com>
Link: http://lkml.kernel.org/r/20150526224512.207378134@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2015-06-19 15:18:28 +02:00
..
2015-04-22 09:24:55 -07:00
2014-09-22 16:48:47 +09:00
2014-12-14 14:57:16 -08:00
2015-03-29 16:17:10 +01:00
2014-11-24 17:24:08 -05:00
2015-04-26 13:36:02 -07:00
2015-02-11 10:28:45 -08:00
2015-01-29 10:49:28 +02:00
2015-04-23 15:16:37 -04:00
2015-04-07 11:17:34 +02:00
2015-04-24 08:23:45 -07:00
2015-03-25 20:28:11 -04:00
2015-04-17 09:03:53 -04:00
2015-05-04 13:09:55 -06:00
2015-04-17 08:31:12 -06:00
2015-03-11 15:56:59 -07:00
2015-04-16 12:33:51 +09:30
2015-02-12 18:54:15 -08:00
2014-12-19 22:55:06 +01:00
2015-01-21 19:21:30 +01:00
2014-11-10 09:27:30 -07:00
2015-03-02 16:43:41 -05:00
2014-12-31 13:06:50 -05:00
2014-10-09 11:35:48 +03:00
2014-10-08 16:01:41 -04:00
2015-05-14 17:55:51 -07:00
2015-02-18 15:08:33 +01:00
2015-06-19 15:18:28 +02:00
2015-03-16 21:45:54 +11:00
2015-04-03 16:18:02 +02:00
2015-01-27 11:09:13 +01:00
2014-09-29 15:37:01 -04:00
2015-04-21 09:42:58 -07:00
2015-04-11 11:17:28 +02:00
2015-01-15 10:34:54 +01:00
2015-01-15 10:34:54 +01:00
2015-02-13 21:21:36 -08:00
2015-01-24 10:11:12 +01:00
2014-12-18 09:39:51 +01:00
2015-01-04 23:11:43 -05:00
2015-05-14 17:55:51 -07:00
2015-04-14 16:49:05 -07:00
2015-03-25 11:44:52 +01:00
2015-04-17 09:04:07 -04:00
2014-10-09 22:25:58 -04:00
2015-03-31 12:04:12 +02:00
2015-04-26 17:33:59 -07:00
2015-04-23 14:36:28 -04:00
2015-03-11 17:56:28 -04:00
2014-11-04 13:29:38 +00:00
2015-04-24 08:46:18 -07:00
2015-04-16 13:53:32 -05:00
2015-05-18 17:17:42 +02:00
2015-04-12 21:03:31 +02:00
2015-01-25 23:17:28 -05:00
2015-02-17 14:34:52 -08:00
2015-01-06 11:01:13 -08:00
2015-03-26 18:18:20 +01:00
2015-04-25 15:49:49 -04:00
2015-02-13 21:21:41 -08:00
2015-03-11 21:55:28 -04:00
2015-04-11 15:53:35 -04:00
2015-01-03 14:32:57 -05:00
2015-04-16 19:02:04 -04:00
2015-02-13 21:21:35 -08:00
2015-03-25 13:28:58 +01:00
2015-04-29 17:10:37 -04:00
2015-05-22 10:36:29 -07:00
2015-05-10 19:26:37 +02:00
2015-04-11 22:29:44 -04:00
2014-11-28 16:08:16 +01:00
2015-04-14 18:06:47 -07:00
2014-12-09 12:05:24 +02:00
2014-12-15 23:49:28 +02:00
2015-03-24 09:48:14 -07:00
2015-04-15 16:35:20 -07:00