linux-stable/arch
Alexei Starovoitov 142fd4d2dc bpf: Add x86-64 JIT support for bpf_addr_space_cast instruction.
LLVM generates bpf_addr_space_cast instruction while translating
pointers between native (zero) address space and
__attribute__((address_space(N))).
The addr_space=1 is reserved as bpf_arena address space.

rY = addr_space_cast(rX, 0, 1) is processed by the verifier and
converted to normal 32-bit move: wX = wY

rY = addr_space_cast(rX, 1, 0) has to be converted by JIT:

aux_reg = upper_32_bits of arena->user_vm_start
aux_reg <<= 32
wX = wY // clear upper 32 bits of dst register
if (wX) // if not zero add upper bits of user_vm_start
  wX |= aux_reg

JIT can do it more efficiently:

mov dst_reg32, src_reg32  // 32-bit move
shl dst_reg, 32
or dst_reg, user_vm_start
rol dst_reg, 32
xor r11, r11
test dst_reg32, dst_reg32 // check if lower 32-bit are zero
cmove r11, dst_reg	  // if so, set dst_reg to zero
			  // Intel swapped src/dst register encoding in CMOVcc

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20240308010812.89848-5-alexei.starovoitov@gmail.com
2024-03-11 15:37:24 -07:00
..
alpha RTC for 6.8 2024-01-18 17:25:39 -08:00
arc work around gcc bugs with 'asm goto' with outputs 2024-02-09 15:57:48 -08:00
arm mm: Introduce vmap_page_range() to map pages in PCI address space 2024-03-11 16:58:10 +01:00
arm64 arm64, bpf: Use bpf_prog_pack for arm64 bpf trampoline 2024-03-09 08:39:55 -08:00
csky work around gcc bugs with 'asm goto' with outputs 2024-02-09 15:57:48 -08:00
hexagon asm-generic cleanups for 6.8 2024-01-10 18:13:44 -08:00
loongarch mm: Introduce vmap_page_range() to map pages in PCI address space 2024-03-11 16:58:10 +01:00
m68k kbuild: use 4-space indentation when followed by conditionals 2024-02-15 06:05:44 +09:00
microblaze Microblaze patches for 6.8-rc1 2024-01-17 11:27:23 -08:00
mips mm: Introduce vmap_page_range() to map pages in PCI address space 2024-03-11 16:58:10 +01:00
nios2 percpu: 2024-01-18 15:01:28 -08:00
openrisc OpenRISC updates for 6.6 2023-09-05 10:09:31 -07:00
parisc parisc architecture fixes for kernel v6.8-rc6: 2024-02-23 10:40:20 -08:00
powerpc mm: Introduce vmap_page_range() to map pages in PCI address space 2024-03-11 16:58:10 +01:00
riscv bpf, riscv64/cfi: Support kCFI + BPF on riscv64 2024-03-06 15:18:16 -08:00
s390 s390 fixes for 6.8-rc6 2024-02-23 09:54:13 -08:00
sh sh updates for v6.8 2024-01-20 09:24:06 -08:00
sparc A list handling fix and 64bit division on 32bit platform fix for the 2024-02-23 08:09:50 +10:00
um work around gcc bugs with 'asm goto' with outputs 2024-02-09 15:57:48 -08:00
x86 bpf: Add x86-64 JIT support for bpf_addr_space_cast instruction. 2024-03-11 15:37:24 -07:00
xtensa work around gcc bugs with 'asm goto' with outputs 2024-02-09 15:57:48 -08:00
.gitignore
Kconfig scs: add CONFIG_MMU dependency for vfree_atomic() 2024-01-25 23:52:21 -08:00