Kernel/linux-stable
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git synced 2025-01-17 18:56:24 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
linux-stable/tools/testing/selftests/rseq/rseq-x86.h

235 lines
7.6 KiB
C
Raw Normal View History

rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
/* SPDX-License-Identifier: LGPL-2.1 OR MIT */
/*
* rseq-x86.h
*
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
* (C) Copyright 2016-2022 - Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
*/
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
#ifndef RSEQ_H
#error "Never use <rseq-x86.h> directly; include <rseq.h> instead."
#endif
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#include <stdint.h>
rseq/selftests: x86: use ud1 instruction as RSEQ_SIG opcode Use ud1 as the guard instruction for the restartable sequence abort handler. Its benefit compared to nopl is to trap execution if the program ends up trying to execute it by mistake, which makes debugging easier. The 4-byte signature per se is unchanged (it is the instruction operand). Only the opcode is changed from nopl to ud1. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Suggested-by: Peter Zijlstra <peterz@infradead.org> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:57 -04:00
/*
* RSEQ_SIG is used with the following reserved undefined instructions, which
* trap in user-space:
*
* x86-32: 0f b9 3d 53 30 05 53 ud1 0x53053053,%edi
* x86-64: 0f b9 3d 53 30 05 53 ud1 0x53053053(%rip),%edi
*/
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#define RSEQ_SIG 0x53053053
rseq/selftests: x86: Work-around bogus gcc-8 optimisation gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance: __thread int var; static int fct(void) { asm goto ( "jmp %l[testlabel]\n\t" : : [var] "m" (var) : : testlabel); return 0; testlabel: return 1; } int main() { return fct(); } % gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status % gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand. Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193 Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Ingo Molnar <mingo@redhat.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:52 -04:00
/*
* Due to a compiler optimization bug in gcc-8 with asm goto and TLS asm input
* operands, we cannot use "m" input operands, and rather pass the __rseq_abi
* address through a "r" input operand.
*/
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
/* Offset of cpu_id, rseq_cs, and mm_cid fields in struct rseq. */
rseq/selftests: x86: Work-around bogus gcc-8 optimisation gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance: __thread int var; static int fct(void) { asm goto ( "jmp %l[testlabel]\n\t" : : [var] "m" (var) : : testlabel); return 0; testlabel: return 1; } int main() { return fct(); } % gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status % gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand. Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193 Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Ingo Molnar <mingo@redhat.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:52 -04:00
#define RSEQ_CPU_ID_OFFSET 4
#define RSEQ_CS_OFFSET 8
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
#define RSEQ_MM_CID_OFFSET 24
rseq/selftests: x86: Work-around bogus gcc-8 optimisation gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance: __thread int var; static int fct(void) { asm goto ( "jmp %l[testlabel]\n\t" : : [var] "m" (var) : : testlabel); return 0; testlabel: return 1; } int main() { return fct(); } % gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status % gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand. Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193 Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Ingo Molnar <mingo@redhat.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:52 -04:00
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#ifdef __x86_64__
selftests/rseq: x86-64: use %fs segment selector for accessing rseq thread area Rather than use rseq_get_abi() and pass its result through a register to the inline assembler, directly access the per-thread rseq area through a memory reference combining the %fs segment selector, the constant offset of the field in struct rseq, and the rseq_offset value (in a register). Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20220124171253.22072-15-mathieu.desnoyers@efficios.com
2022-01-24 12:12:52 -05:00
#define RSEQ_ASM_TP_SEGMENT %%fs
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#define rseq_smp_mb() \
__asm__ __volatile__ ("lock; addl $0,-128(%%rsp)" ::: "memory", "cc")
#define rseq_smp_rmb() rseq_barrier()
#define rseq_smp_wmb() rseq_barrier()
#define rseq_smp_load_acquire(p) \
__extension__ ({ \
selftests/rseq: Use rseq_unqual_scalar_typeof in macros Use rseq_unqual_scalar_typeof() rather than typeof() in macros to remove the volatile qualifier (if there is one in the input argument), thus generating better assembly code in those scenarios. Also add extra brackets around the "p" parameter in RSEQ_READ_ONCE(), RSEQ_WRITE_ONCE(), and rseq_unqual_scalar_typeof() across architectures to preserve expectations of operator priority. Here is an example that shows how operator priority may be an issue with missing parentheses: #define m(p) \ do { \ __typeof__(*p) v = 0; \ } while (0) void fct(unsigned long long *p1) { m(p1 + 1); /* works */ m(1 + p1); /* broken */ } Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-06-27 11:29:23 -04:00
rseq_unqual_scalar_typeof(*(p)) ____p1 = RSEQ_READ_ONCE(*(p)); \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
rseq_barrier(); \
____p1; \
})
#define rseq_smp_acquire__after_ctrl_dep() rseq_smp_rmb()
#define rseq_smp_store_release(p, v) \
do { \
rseq_barrier(); \
selftests/rseq: Use rseq_unqual_scalar_typeof in macros Use rseq_unqual_scalar_typeof() rather than typeof() in macros to remove the volatile qualifier (if there is one in the input argument), thus generating better assembly code in those scenarios. Also add extra brackets around the "p" parameter in RSEQ_READ_ONCE(), RSEQ_WRITE_ONCE(), and rseq_unqual_scalar_typeof() across architectures to preserve expectations of operator priority. Here is an example that shows how operator priority may be an issue with missing parentheses: #define m(p) \ do { \ __typeof__(*p) v = 0; \ } while (0) void fct(unsigned long long *p1) { m(p1 + 1); /* works */ m(1 + p1); /* broken */ } Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-06-27 11:29:23 -04:00
RSEQ_WRITE_ONCE(*(p), v); \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
} while (0)
#define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \
start_ip, post_commit_offset, abort_ip) \
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:54 -04:00
".pushsection __rseq_cs, \"aw\"\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
".balign 32\n\t" \
__rseq_str(label) ":\n\t" \
".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:54 -04:00
".popsection\n\t" \
".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \
".quad " __rseq_str(label) "b\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
".popsection\n\t"
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:54 -04:00
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \
__RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \
(post_commit_ip - start_ip), abort_ip)
rseq/selftests: Add __rseq_exit_point_array section for debuggers Knowing all exit points is useful to assist debuggers stepping over the rseq critical sections without requiring them to disassemble the content of the critical section to figure out the exit points. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:53 -04:00
/*
* Exit points of a rseq critical section consist of all instructions outside
* of the critical section where a critical section can either branch to or
* reach through the normal course of its execution. The abort IP and the
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:54 -04:00
* post-commit IP are already part of the __rseq_cs section and should not be
* explicitly defined as additional exit points. Knowing all exit points is
rseq/selftests: Add __rseq_exit_point_array section for debuggers Knowing all exit points is useful to assist debuggers stepping over the rseq critical sections without requiring them to disassemble the content of the critical section to figure out the exit points. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:53 -04:00
* useful to assist debuggers stepping over the critical section.
*/
#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \
".pushsection __rseq_exit_point_array, \"aw\"\n\t" \
".quad " __rseq_str(start_ip) ", " __rseq_str(exit_ip) "\n\t" \
".popsection\n\t"
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \
RSEQ_INJECT_ASM(1) \
"leaq " __rseq_str(cs_label) "(%%rip), %%rax\n\t" \
rseq/selftests: x86: Work-around bogus gcc-8 optimisation gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance: __thread int var; static int fct(void) { asm goto ( "jmp %l[testlabel]\n\t" : : [var] "m" (var) : : testlabel); return 0; testlabel: return 1; } int main() { return fct(); } % gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status % gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand. Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193 Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Ingo Molnar <mingo@redhat.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:52 -04:00
"movq %%rax, " __rseq_str(rseq_cs) "\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
__rseq_str(label) ":\n\t"
#define RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, label) \
RSEQ_INJECT_ASM(2) \
rseq/selftests: x86: Work-around bogus gcc-8 optimisation gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance: __thread int var; static int fct(void) { asm goto ( "jmp %l[testlabel]\n\t" : : [var] "m" (var) : : testlabel); return 0; testlabel: return 1; } int main() { return fct(); } % gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status % gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand. Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193 Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Ingo Molnar <mingo@redhat.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:52 -04:00
"cmpl %[" __rseq_str(cpu_id) "], " __rseq_str(current_cpu_id) "\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
"jnz " __rseq_str(label) "\n\t"
#define RSEQ_ASM_DEFINE_ABORT(label, teardown, abort_label) \
".pushsection __rseq_failure, \"ax\"\n\t" \
rseq/selftests: x86: use ud1 instruction as RSEQ_SIG opcode Use ud1 as the guard instruction for the restartable sequence abort handler. Its benefit compared to nopl is to trap execution if the program ends up trying to execute it by mistake, which makes debugging easier. The 4-byte signature per se is unchanged (it is the instruction operand). Only the opcode is changed from nopl to ud1. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Suggested-by: Peter Zijlstra <peterz@infradead.org> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:57 -04:00
/* Disassembler-friendly signature: ud1 <sig>(%rip),%edi. */ \
".byte 0x0f, 0xb9, 0x3d\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
".long " __rseq_str(RSEQ_SIG) "\n\t" \
__rseq_str(label) ":\n\t" \
teardown \
"jmp %l[" __rseq_str(abort_label) "]\n\t" \
".popsection\n\t"
#define RSEQ_ASM_DEFINE_CMPFAIL(label, teardown, cmpfail_label) \
".pushsection __rseq_failure, \"ax\"\n\t" \
__rseq_str(label) ":\n\t" \
teardown \
"jmp %l[" __rseq_str(cmpfail_label) "]\n\t" \
".popsection\n\t"
selftests/rseq: Fix warnings about #if checks of undefined tokens Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20220124171253.22072-12-mathieu.desnoyers@efficios.com
2022-01-24 12:12:49 -05:00
#elif defined(__i386__)
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86-32: use %gs segment selector for accessing rseq thread area Rather than use rseq_get_abi() and pass its result through a register to the inline assembler, directly access the per-thread rseq area through a memory reference combining the %gs segment selector, the constant offset of the field in struct rseq, and the rseq_offset value (in a register). Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20220124171253.22072-16-mathieu.desnoyers@efficios.com
2022-01-24 12:12:53 -05:00
#define RSEQ_ASM_TP_SEGMENT %%gs
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#define rseq_smp_mb() \
__asm__ __volatile__ ("lock; addl $0,-128(%%esp)" ::: "memory", "cc")
#define rseq_smp_rmb() \
__asm__ __volatile__ ("lock; addl $0,-128(%%esp)" ::: "memory", "cc")
#define rseq_smp_wmb() \
__asm__ __volatile__ ("lock; addl $0,-128(%%esp)" ::: "memory", "cc")
#define rseq_smp_load_acquire(p) \
__extension__ ({ \
__typeof(*p) ____p1 = RSEQ_READ_ONCE(*p); \
rseq_smp_mb(); \
____p1; \
})
#define rseq_smp_acquire__after_ctrl_dep() rseq_smp_rmb()
#define rseq_smp_store_release(p, v) \
do { \
rseq_smp_mb(); \
RSEQ_WRITE_ONCE(*p, v); \
} while (0)
/*
* Use eax as scratch register and take memory operands as input to
* lessen register pressure. Especially needed when compiling in O0.
*/
#define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \
start_ip, post_commit_offset, abort_ip) \
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:54 -04:00
".pushsection __rseq_cs, \"aw\"\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
".balign 32\n\t" \
__rseq_str(label) ":\n\t" \
".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
".long " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:54 -04:00
".popsection\n\t" \
".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \
".long " __rseq_str(label) "b, 0x0\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
".popsection\n\t"
#define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \
__RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \
(post_commit_ip - start_ip), abort_ip)
rseq/selftests: Add __rseq_exit_point_array section for debuggers Knowing all exit points is useful to assist debuggers stepping over the rseq critical sections without requiring them to disassemble the content of the critical section to figure out the exit points. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:53 -04:00
/*
* Exit points of a rseq critical section consist of all instructions outside
* of the critical section where a critical section can either branch to or
* reach through the normal course of its execution. The abort IP and the
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:54 -04:00
* post-commit IP are already part of the __rseq_cs section and should not be
* explicitly defined as additional exit points. Knowing all exit points is
rseq/selftests: Add __rseq_exit_point_array section for debuggers Knowing all exit points is useful to assist debuggers stepping over the rseq critical sections without requiring them to disassemble the content of the critical section to figure out the exit points. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:53 -04:00
* useful to assist debuggers stepping over the critical section.
*/
#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \
".pushsection __rseq_exit_point_array, \"aw\"\n\t" \
".long " __rseq_str(start_ip) ", 0x0, " __rseq_str(exit_ip) ", 0x0\n\t" \
".popsection\n\t"
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
#define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \
RSEQ_INJECT_ASM(1) \
rseq/selftests: x86: Work-around bogus gcc-8 optimisation gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance: __thread int var; static int fct(void) { asm goto ( "jmp %l[testlabel]\n\t" : : [var] "m" (var) : : testlabel); return 0; testlabel: return 1; } int main() { return fct(); } % gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status % gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand. Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193 Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Ingo Molnar <mingo@redhat.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:52 -04:00
"movl $" __rseq_str(cs_label) ", " __rseq_str(rseq_cs) "\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
__rseq_str(label) ":\n\t"
#define RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, label) \
RSEQ_INJECT_ASM(2) \
rseq/selftests: x86: Work-around bogus gcc-8 optimisation gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance: __thread int var; static int fct(void) { asm goto ( "jmp %l[testlabel]\n\t" : : [var] "m" (var) : : testlabel); return 0; testlabel: return 1; } int main() { return fct(); } % gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status % gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main': test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2' collect2: error: ld returned 1 exit status Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand. Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193 Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: Ingo Molnar <mingo@redhat.com> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:52 -04:00
"cmpl %[" __rseq_str(cpu_id) "], " __rseq_str(current_cpu_id) "\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
"jnz " __rseq_str(label) "\n\t"
#define RSEQ_ASM_DEFINE_ABORT(label, teardown, abort_label) \
".pushsection __rseq_failure, \"ax\"\n\t" \
rseq/selftests: x86: use ud1 instruction as RSEQ_SIG opcode Use ud1 as the guard instruction for the restartable sequence abort handler. Its benefit compared to nopl is to trap execution if the program ends up trying to execute it by mistake, which makes debugging easier. The 4-byte signature per se is unchanged (it is the instruction operand). Only the opcode is changed from nopl to ud1. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Suggested-by: Peter Zijlstra <peterz@infradead.org> CC: Peter Zijlstra <peterz@infradead.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Joel Fernandes <joelaf@google.com> CC: Catalin Marinas <catalin.marinas@arm.com> CC: Dave Watson <davejwatson@fb.com> CC: Will Deacon <will.deacon@arm.com> CC: Shuah Khan <shuah@kernel.org> CC: Andi Kleen <andi@firstfloor.org> CC: linux-kselftest@vger.kernel.org CC: "H . Peter Anvin" <hpa@zytor.com> CC: Chris Lameter <cl@linux.com> CC: Russell King <linux@arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> CC: Paul Turner <pjt@google.com> CC: Boqun Feng <boqun.feng@gmail.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Ben Maurer <bmaurer@fb.com> CC: linux-api@vger.kernel.org CC: Andy Lutomirski <luto@amacapital.net> CC: Andrew Morton <akpm@linux-foundation.org> CC: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-04-29 11:27:57 -04:00
/* Disassembler-friendly signature: ud1 <sig>,%edi. */ \
".byte 0x0f, 0xb9, 0x3d\n\t" \
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
".long " __rseq_str(RSEQ_SIG) "\n\t" \
__rseq_str(label) ":\n\t" \
teardown \
"jmp %l[" __rseq_str(abort_label) "]\n\t" \
".popsection\n\t"
#define RSEQ_ASM_DEFINE_CMPFAIL(label, teardown, cmpfail_label) \
".pushsection __rseq_failure, \"ax\"\n\t" \
__rseq_str(label) ":\n\t" \
teardown \
"jmp %l[" __rseq_str(cmpfail_label) "]\n\t" \
".popsection\n\t"
#endif
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
/* Per-cpu-id indexing. */
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
#define RSEQ_TEMPLATE_CPU_ID
#define RSEQ_TEMPLATE_MO_RELAXED
#include "rseq-x86-bits.h"
#undef RSEQ_TEMPLATE_MO_RELAXED
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
#define RSEQ_TEMPLATE_MO_RELEASE
#include "rseq-x86-bits.h"
#undef RSEQ_TEMPLATE_MO_RELEASE
#undef RSEQ_TEMPLATE_CPU_ID
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
/* Per-mm-cid indexing. */
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
#define RSEQ_TEMPLATE_MM_CID
#define RSEQ_TEMPLATE_MO_RELAXED
#include "rseq-x86-bits.h"
#undef RSEQ_TEMPLATE_MO_RELAXED
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
#define RSEQ_TEMPLATE_MO_RELEASE
#include "rseq-x86-bits.h"
#undef RSEQ_TEMPLATE_MO_RELEASE
#undef RSEQ_TEMPLATE_MM_CID
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
/* APIs which are not based on cpu ids. */
rseq/selftests: Provide rseq library This rseq helper library provides a user-space API to the rseq() system call. The rseq fast-path exposes the instruction pointer addresses where the rseq assembly blocks begin and end, as well as the associated abort instruction pointer, in the __rseq_table section. This section allows debuggers may know where to place breakpoints when single-stepping through assembly blocks which may be aborted at any point by the kernel. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Joel Fernandes <joelaf@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Dave Watson <davejwatson@fb.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Shuah Khan <shuahkh@osg.samsung.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: linux-kselftest@vger.kernel.org Cc: "H . Peter Anvin" <hpa@zytor.com> Cc: Chris Lameter <cl@linux.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Andrew Hunter <ahh@google.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com> Cc: Paul Turner <pjt@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Josh Triplett <josh@joshtriplett.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Maurer <bmaurer@fb.com> Cc: linux-api@vger.kernel.org Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-02 08:44:04 -04:00
selftests/rseq: x86: Template memory ordering and percpu access mode Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com
2022-11-22 15:39:13 -05:00
#define RSEQ_TEMPLATE_CPU_ID_NONE
#define RSEQ_TEMPLATE_MO_RELAXED
#include "rseq-x86-bits.h"
#undef RSEQ_TEMPLATE_MO_RELAXED
#undef RSEQ_TEMPLATE_CPU_ID_NONE
Reference in New Issue Copy Permalink