Eric Dumazet
889a7a6a5d
percpu, x86: Fix percpu_xchg_op()
...
These recent percpu commits:
2485b6464cf8: x86,percpu: Move out of place 64 bit ops into X86_64 section
8270137a0d50: cpuops: Use cmpxchg for xchg to avoid lock semantics
Caused this 'perf top' crash:
Kernel panic - not syncing: Fatal exception in interrupt
Pid: 0, comm: swapper Tainted: G D
2.6.38-rc2-00181-gef71723 #413 Call Trace: <IRQ> [<ffffffff810465b5>]
? panic
? kmsg_dump
? kmsg_dump
? oops_end
? no_context
? __bad_area_nosemaphore
? perf_output_begin
? bad_area_nosemaphore
? do_page_fault
? __task_pid_nr_ns
? perf_event_tid
? __perf_event_header__init_id
? validate_chain
? perf_output_sample
? trace_hardirqs_off
? page_fault
? irq_work_run
? update_process_times
? tick_sched_timer
? tick_sched_timer
? __run_hrtimer
? hrtimer_interrupt
? account_system_vtime
? smp_apic_timer_interrupt
? apic_timer_interrupt
...
Looking at assembly code, I found:
list = this_cpu_xchg(irq_work_list, NULL);
gives this wrong code : (gcc-4.1.2 cross compiler)
ffffffff810bc45e:
mov %gs:0xead0,%rax
cmpxchg %rax,%gs:0xead0
jne ffffffff810bc45e <irq_work_run+0x3e>
test %rax,%rax
je ffffffff810bc4aa <irq_work_run+0x8a>
Tell gcc we dirty eax/rax register in percpu_xchg_op()
Compiler must use another register to store pxo_new__
We also dont need to reload percpu value after a jump,
since a 'failed' cmpxchg already updated eax/rax
Wrong generated code was :
xor %rax,%rax /* load 0 into %rax */
1: mov %gs:0xead0,%rax
cmpxchg %rax,%gs:0xead0
jne 1b
test %rax,%rax
After patch :
xor %rdx,%rdx /* load 0 into %rdx */
mov %gs:0xead0,%rax
1: cmpxchg %rdx,%gs:0xead0
jne 1b:
test %rax,%rax
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Tejun Heo <tj@kernel.org>
LKML-Reference: <1295973114.3588.312.camel@edumazet-laptop>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2011-01-26 08:10:49 +01:00
..
2011-01-03 20:35:03 +01:00
2010-02-19 16:12:29 -08:00
2011-01-13 18:46:48 -08:00
2009-11-10 11:23:05 +01:00
2008-10-22 22:55:23 -07:00
2010-12-23 15:27:16 -08:00
2009-02-18 16:48:05 +08:00
2009-08-03 09:05:00 +01:00
2010-04-28 17:15:47 -07:00
2011-01-06 11:11:50 -08:00
2010-10-13 11:13:21 +02:00
2010-10-13 11:13:21 +02:00
2010-10-21 14:23:48 -07:00
2011-01-11 12:43:41 +01:00
2010-08-12 14:01:38 -07:00
2011-01-04 09:43:42 +01:00
2010-12-23 13:15:53 -08:00
2009-01-29 14:16:51 +01:00
2010-05-17 15:17:16 -07:00
2009-12-12 13:08:14 +01:00
2009-08-31 15:14:30 -07:00
2010-02-25 20:47:30 -08:00
2010-05-18 08:40:05 -07:00
2010-05-18 08:40:05 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2010-10-09 21:51:44 +02:00
2009-06-11 21:02:14 +02:00
2011-01-13 08:03:25 -08:00
2010-11-12 00:45:41 +01:00
2009-12-05 09:10:12 -08:00
2008-10-22 22:55:23 -07:00
2009-01-14 19:56:50 -08:00
2010-03-03 11:26:00 +01:00
2011-01-21 14:11:12 +01:00
2010-08-26 15:13:13 -07:00
2010-10-19 14:28:02 -07:00
2009-10-01 16:11:12 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:20 -07:00
2010-07-28 17:05:11 -07:00
2010-07-28 15:24:09 -07:00
2008-10-22 22:55:20 -07:00
2010-09-14 16:08:45 -07:00
2011-01-21 18:14:54 -08:00
2010-10-21 13:01:08 -07:00
2009-03-13 14:49:54 +10:30
2008-10-22 22:55:20 -07:00
2009-08-04 01:28:52 +09:00
2010-12-30 12:20:28 +01:00
2008-10-22 22:55:23 -07:00
2009-12-04 15:39:55 +01:00
2009-11-05 13:22:18 -08:00
2009-11-27 14:20:32 +01:00
2008-10-22 22:55:23 -07:00
2010-08-11 08:59:21 -07:00
2008-10-22 22:55:23 -07:00
2009-03-23 17:20:50 +01:00
2010-09-03 08:14:11 +02:00
2010-12-17 10:01:30 -08:00
2008-10-22 22:55:23 -07:00
2010-08-27 11:13:47 -07:00
2010-02-16 08:51:49 -08:00
2008-11-11 16:19:48 -08:00
2010-10-23 08:25:36 -07:00
2008-10-22 22:55:20 -07:00
2010-02-16 21:22:26 -08:00
2008-10-22 22:55:20 -07:00
2011-01-11 12:46:16 +01:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:20 -07:00
2009-08-26 21:30:39 +02:00
2008-10-22 22:55:23 -07:00
2010-10-21 14:23:48 -07:00
2009-02-17 17:52:43 +01:00
2009-12-15 08:53:28 -08:00
2011-01-11 12:46:15 +01:00
2010-10-18 19:58:50 +02:00
2010-10-26 16:52:08 -07:00
2010-10-12 16:53:37 +02:00
2008-10-22 22:55:23 -07:00
2010-09-17 03:24:13 +02:00
2010-10-21 14:11:46 -07:00
2008-10-22 22:55:23 -07:00
2010-05-07 17:13:04 -07:00
2011-01-07 10:03:50 -05:00
2010-10-22 15:33:38 -07:00
2010-03-02 10:28:38 +01:00
2010-10-12 16:53:36 +02:00
2008-10-22 22:55:23 -07:00
2009-03-27 14:43:57 -04:00
2008-12-12 11:58:36 +01:00
2009-08-27 00:35:56 +02:00
2009-10-29 08:47:46 +01:00
2009-03-05 14:17:18 +01:00
2010-03-10 13:23:34 +01:00
2010-03-24 21:37:57 +08:00
2010-08-03 09:50:30 -04:00
2011-01-04 09:43:42 +01:00
2010-10-28 17:11:17 -07:00
2008-10-22 22:55:20 -07:00
2009-06-18 14:40:03 -07:00
2010-10-26 16:52:08 -07:00
2010-10-08 13:11:21 -07:00
2009-11-15 09:03:10 +01:00
2009-06-18 14:40:03 -07:00
2009-02-17 17:52:44 +01:00
2009-01-21 17:26:06 +09:00
2010-10-12 16:53:42 +02:00
2010-10-18 19:58:50 +02:00
2011-01-13 10:15:12 -08:00
2010-10-07 14:08:55 +01:00
2008-10-22 22:55:23 -07:00
2011-01-23 16:12:45 +01:00
2010-08-14 22:26:51 +02:00
2011-01-07 15:08:53 +01:00
2009-03-10 18:13:25 -07:00
2010-08-05 09:22:20 -05:00
2009-06-16 19:47:51 -07:00
2009-06-13 15:37:30 +02:00
2010-03-10 13:23:34 +01:00
2011-01-12 11:31:07 +02:00
2011-01-13 17:32:46 -08:00
2011-01-12 11:23:27 +02:00
2010-08-01 10:46:37 +03:00
2008-10-22 22:55:23 -07:00
2010-04-14 21:43:56 +09:30
2009-08-25 15:40:29 -07:00
2009-03-11 12:39:28 +01:00
2010-06-09 11:12:36 +02:00
2010-01-05 15:34:49 +09:00
2009-01-29 14:16:51 +01:00
2011-01-07 15:08:51 +01:00
2009-02-10 00:39:14 +01:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2011-01-03 08:30:30 -08:00
2010-10-05 21:44:35 -07:00
2010-11-10 14:54:54 +01:00
2009-06-18 14:40:03 -07:00
2008-10-22 22:55:23 -07:00
2009-09-24 09:34:52 +09:30
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2009-12-04 15:39:55 +01:00
2010-02-15 14:34:10 -08:00
2008-10-22 22:55:20 -07:00
2010-06-29 12:12:59 +02:00
2011-01-05 14:09:23 +01:00
2011-01-05 14:09:23 +01:00
2010-11-11 11:34:27 +01:00
2010-11-11 11:34:27 +01:00
2009-06-18 14:40:03 -07:00
2010-05-07 17:13:04 -07:00
2009-03-17 15:38:59 -07:00
2011-01-06 11:11:50 -08:00
2010-07-20 15:38:18 -07:00
2009-08-21 17:00:02 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:20 -07:00
2010-09-17 15:36:40 -07:00
2011-01-07 15:08:52 +01:00
2009-09-10 17:22:44 -04:00
2011-01-19 10:25:18 +01:00
2011-01-19 10:25:18 +01:00
2008-10-22 22:55:20 -07:00
2010-03-07 15:59:39 -08:00
2010-12-15 17:11:30 -08:00
2011-01-13 08:03:13 -08:00
2010-06-29 12:12:59 +02:00
2009-02-13 13:36:47 +01:00
2009-06-20 15:40:00 -07:00
2009-02-11 14:54:09 -08:00
2010-10-07 16:36:17 -07:00
2010-08-11 08:43:49 +02:00
2009-06-18 14:40:03 -07:00
2011-01-13 17:32:39 -08:00
2011-01-13 17:32:39 -08:00
2008-10-22 22:55:23 -07:00
2009-11-23 17:09:59 -08:00
2010-02-10 17:47:18 -08:00
2010-10-18 10:49:30 -04:00
2008-10-22 22:55:23 -07:00
2009-01-29 14:16:51 +01:00
2010-11-18 12:52:04 +01:00
2011-01-26 08:10:49 +01:00
2011-01-09 10:40:52 +01:00
2010-11-26 15:14:56 +01:00
2010-11-01 15:38:34 -04:00
2010-06-07 17:27:11 -07:00
2010-10-26 16:52:08 -07:00
2009-05-05 19:10:18 -07:00
2011-01-13 17:32:44 -08:00
2011-01-13 17:32:44 -08:00
2009-02-13 11:35:01 -08:00
2011-01-13 17:32:44 -08:00
2009-02-13 11:35:01 -08:00
2011-01-13 17:32:44 -08:00
2011-01-13 17:32:45 -08:00
2008-10-22 22:55:20 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:20 -07:00
2009-02-02 23:27:09 +05:30
2008-10-22 22:55:20 -07:00
2008-10-22 22:55:23 -07:00
2011-01-13 20:15:18 -08:00
2010-12-15 17:11:30 -08:00
2010-02-12 09:42:39 -08:00
2010-03-26 11:33:55 +01:00
2010-03-26 11:33:55 +01:00
2010-05-19 11:41:05 +03:00
2010-11-28 09:33:20 +01:00
2008-10-22 22:55:23 -07:00
2008-11-12 18:55:46 +01:00
2010-07-07 17:29:18 -07:00
2008-10-22 22:55:20 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:20 -07:00
2008-10-22 22:55:23 -07:00
2010-07-20 17:41:14 -07:00
2010-08-07 18:15:50 +02:00
2009-03-02 15:41:30 -08:00
2009-03-02 15:41:30 -08:00
2008-10-22 22:55:20 -07:00
2009-10-20 14:46:00 +09:00
2010-10-19 14:28:02 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2009-01-29 14:16:51 +01:00
2010-11-12 00:45:41 +01:00
2009-06-18 14:40:03 -07:00
2008-10-22 22:55:23 -07:00
2009-01-31 00:18:58 +05:30
2009-12-04 23:41:47 +01:00
2008-12-18 15:01:22 -08:00
2008-10-22 22:55:23 -07:00
2009-06-11 21:01:52 +02:00
2010-10-21 13:30:44 -07:00
2010-11-18 09:08:23 +01:00
2009-06-18 14:40:03 -07:00
2009-06-18 14:40:03 -07:00
2009-05-05 19:10:18 -07:00
2009-12-14 23:55:32 +01:00
2009-12-14 23:55:32 +01:00
2008-10-22 22:55:23 -07:00
2009-09-03 21:30:51 +02:00
2010-11-18 14:37:34 +01:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2009-09-28 16:43:15 -07:00
2009-06-15 12:40:02 +02:00
2008-10-22 22:55:20 -07:00
2010-06-08 00:32:49 +02:00
2010-06-08 00:32:49 +02:00
2008-10-22 22:55:20 -07:00
2011-01-12 11:31:07 +02:00
2009-01-31 00:19:32 +05:30
2010-08-26 15:13:29 -07:00
2008-10-22 22:55:23 -07:00
2010-08-13 16:53:13 -07:00
2010-02-17 13:07:21 +01:00
2010-08-17 18:07:43 -07:00
2010-08-04 11:47:58 -07:00
2008-10-22 22:55:23 -07:00
2009-06-18 14:40:03 -07:00
2009-06-18 14:40:03 -07:00
2010-05-28 16:14:17 -07:00
2009-09-16 14:34:50 +02:00
2010-11-18 09:08:23 +01:00
2009-06-16 19:47:27 -07:00
2008-10-22 22:55:23 -07:00
2010-10-20 14:23:55 -07:00
2010-05-27 09:12:57 -07:00
2010-10-20 14:23:55 -07:00
2011-01-12 11:23:16 +02:00
2010-08-20 14:59:02 +02:00
2009-06-18 14:40:03 -07:00
2010-01-05 13:45:06 -08:00
2010-02-28 10:35:09 -08:00
2009-10-12 18:29:46 +02:00
2009-06-18 14:40:03 -07:00
2008-10-22 22:55:23 -07:00
2010-08-10 12:07:51 -07:00
2010-08-10 12:07:51 -07:00
2008-10-22 22:55:20 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2010-02-11 15:08:17 -08:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2009-08-21 21:43:46 +02:00
2009-03-24 11:02:46 +02:00
2008-10-22 22:55:23 -07:00
2011-01-12 11:31:08 +02:00
2008-10-22 22:55:23 -07:00
2010-10-18 10:49:34 -04:00
2008-10-22 22:55:20 -07:00
2008-10-22 22:55:23 -07:00
2008-10-22 22:55:23 -07:00
2009-06-15 12:40:02 +02:00
2010-08-06 16:25:13 -07:00