Provide the necessary infrastructure to trap coprocessor accesses that
occur when running 32bit guests.
Also wire SMC and HVC trapped in 32bit mode while were at it.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
As conditional instructions can trap on AArch32, add the thinest
possible emulation layer to keep 32bit guests happy.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Allow access to the 32bit register file through the usual API.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Define the 32bit specific registers (SPSRs, cp15...).
Most CPU registers are directly mapped to a 64bit register
(r0->x0...). Only the SPSRs have separate registers.
cp15 registers are also mapped into their 64bit counterpart in most
cases.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Only the Makefile is plugged in. The Kconfig stuff is in a separate
patch to allow for an easier merge process.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Wire the PSCI backend into the exit handling code.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Add support for the in-kernel timer emulation.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Add support for the in-kernel GIC emulation.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Handle the exit of a VM, decoding the exit reason from HYP mode
and calling the corresponding handler.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The HYP mode world switch in all its glory.
Implements save/restore of host/guest registers, EL2 trapping,
IPA resolution, and additional services (tlb invalidation).
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iQIcBAABAgAGBQJRsbjNAAoJEIlPj0hw4a6Q3wYP/i6YVChCoYRhUWqWt4wRnw8I
bWgwPXWwfc+nJLNP4nwp2ULro/f2hu0KOWZjl/HahPUKt88rD/e0y+bSb4nm4R2h
/NzuMB3EllwAqeW7a1xOLzAB4y9h0t0iYVhg3joJ+sHaZk9hX3NZ2InmhniFCgaD
cQFAXCPkpuLYxm9PClD/LCCx4UoZxz5BewM4DujSoK7jTcRFPyrivM/HLpgkgVXx
1onHTzH4m0iS7INSD5A28tmUvk3G/TwomkO4C55Iu6l4fSUjegmGMmWQvk6cWBoz
3UHI8YbG7pL4f/EKZAJdeK3Q/SaR1+V0q4IY+OJwPZUtBlNmXfCFUsljWqSw5kye
LzhA4zmJOSYYr4g+wRE228IzxZsw8izvH/+bM/7gLcMJfd535tVoVYvxmMj5L9mh
e1hb9AS4BvZNVzCyzb74x2NjscE55k9CgnK35DRSxr42Faap8c34Q43fYy2OWVX/
4NfsH6NovQNmaxwTM4Y/G0c9zagFbTpxM4tuJvBqkc9eeuX22mHoGmzNQkXkGIA4
cqjvgLNjqMnfFS9Wzf1lDIYwcxJFfCgukDaNFW28wLhXpj73hMW/maIlnuzjLEEA
ZRA28VSaGGnxckYTBo0LBs5uW0Ef+la676cOp96O4Ei9AFHladej1p+sXKqIQZ0I
xpyN01RtUy439hGKaNSx
=uMi/
-----END PGP SIGNATURE-----
Merge tag 'xen-arm64-3.1-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen into upstream
Introduce Xen support to ARM64
* tag 'xen-arm64-3.1-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen:
MAINTAINERS: add myself as arm64/xen maintainer
arm64/xen: introduce CONFIG_XEN and hypercall.S on ARM64
arm64/xen: use XEN_IO_PROTO_ABI_ARM on ARM64
arm64/xen: implement ioremap_cached on arm64
arm64/xen: introduce asm/xen header files on arm64
arm/xen: define xen_remap as ioremap_cached
The software breakpoint handlers are hooked in directly from ptrace,
which makes it difficult to add additional handlers for things like
kprobes and kgdb.
This patch moves the handling code into debug-monitors.c, where we can
dispatch to different debug subsystems more easily.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
As is done for other architectures, sort the exception table at
build-time rather than during boot.
Since sortextable appears to be a standalone C program relying on the
host elf.h to provide EM_AARCH64, I've had to add a conditional check in
order to allow cross-compilation on machines that aren't running a
bleeding-edge libc-dev.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
When using an IOMMU for device mappings, it is necessary to keep a
pointer between the device and the IOMMU to which it is attached in
order to obtain the correct IOMMU when attaching the device to a domain.
This patch adds an iommu pointer to the dev_archdata structure, in a
similar manner to other architectures (ARM, PowerPC, x86, ...).
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
pte_index is a useful helper outside of arch/arm64, for things like the
ARM SMMU driver, so rename __pte_index to pte_index to be consistent
with both arch/arm/ and also the definitions of pmd_index and pgd_index.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Under arm64, we will calibrate the delay loop statically using a known
timer frequency, so delete read_current_timer(), or it will cause
compiling issue with allmodconfig.
The related error:
ERROR: "read_current_timer" [lib/rbtree_test.ko] undefined!
ERROR: "read_current_timer" [lib/interval_tree_test.ko] undefined!
ERROR: "read_current_timer" [fs/ext4/ext4.ko] undefined!
ERROR: "read_current_timer" [crypto/tcrypt.ko] undefined!
Signed-off-by: Chen Gang <gang.chen@asianux.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
We don't support software broadcast of cache maintenance operations, so
this flush is not required (__sync_icache_dcache will always affect all
CPUs).
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
An exclusive store instruction may fail for reasons other than lock
contention (e.g. a cache eviction during the critical section) so, in
line with other architectures using similar exclusive instructions
(alpha, mips, powerpc), retry the trylock operation if the lock appears
to be free but the strex reported failure.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Tony Thompson <anthony.thompson@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
This function is only used in __sync_icache_dcache(), so remove it and
call __flush_dcache_area() directly. The flush_icache_user_range()
function is not used in the arm64 kernel.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Will Deacon <will.deacon@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
The D-cache on AArch64 is VIPT non-aliasing, so there is no need to
flush it for anonymous pages.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Will Deacon <will.deacon@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
The flush_dcache_page() function is called when the kernel modified a
page cache page. Since the D-cache on AArch64 does not have aliases
this function can simply mark the page as dirty for later flushing via
set_pte_at()/__sync_icache_dcache() if the page is executable (to ensure
the I-D cache coherency).
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Will Deacon <will.deacon@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Most architectures that define CONFIG_HAS_DMA, have implementations for
both dma_alloc_attrs() and dma_free_attrs(). All achitectures that do
not define CONFIG_HAS_DMA also have both of these definitions provided
by dma-mapping-broken.h.
Add default implementations for these functions on arm64.
Signed-off-by: Damian Hobson-Garcia <dhobsong@igel.co.jp>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Provide EL2 with page tables and stack, and set the vectors
to point to the full blown world-switch code.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Let userspace play with the guest registers.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Define the necessary structures to perform an MMIO access.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Provide the architecture dependent structures for VM and
vcpu abstractions.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Provide the reset code for a virtual CPU booted in 64bit mode.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Add the support code for CPU specific system registers. Not much
here yet.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Provide 64bit system register handling, modeled after the cp15
handling for ARM.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Provide the kvm.h file that defines the user space visible
interface.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Implement the injection of a fault (undefined, data abort or
prefetch abort) into a 64bit guest.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Implements helpers for dealing with the EL2 syndrome register as
well as accessing the vcpu registers.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Define the saved/restored registers for 64bit guests.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Define all the useful bitfields for EL2 registers.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Add the necessary infrastructure for identity-mapped HYP page
tables. Idmap-ed code must be in the ".hyp.idmap.text" linker
section.
The rest of the HYP ends up in ".hyp.text".
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Add HYP and S2 page flags, for both normal and device memory.
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Introduce CONFIG_XEN and the implementation of hypercall.S (that is
the only ARMv8 specific code in Xen support for ARM).
Compile enlighten.c and grant_table.c from arch/arm.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
asm/xen/hypercall.h, asm/xen/hypervisor.h, asm/xen/interface.h and
asm/xen/page.h are identical so to avoid code duplication we are just
including the original arm header files here.
asm/xen/events.h is slightly different, so introduce a different file
here (use xchg to implement xchg_xen_ulong and pass regs->pstate to
raw_irqs_disabled_flags).
Also introduce asm/hypervisor.h and asm/sync_bitops.h.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Switching between reading the virtual or physical counters is
problematic, as some core code wants a view of time before we're fully
set up. Using a function pointer and switching the source after the
first read can make time appear to go backwards, and having a check in
the read function is an unfortunate block on what we want to be a fast
path.
Instead, this patch makes us always use the virtual counters. If we're a
guest, or don't have hyp mode, we'll use the virtual timers, and as such
don't care about CNTVOFF as long as it doesn't change in such a way as
to make time appear to travel backwards. As the guest will use the
virtual timers, a (potential) KVM host must use the physical timers
(which can wake up the host even if they fire while a guest is
executing), and hence a host must have CNTVOFF set to zero so as to have
a consistent view of time between the physical timers and virtual
counters.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Rob Herring <rob.herring@calxeda.com>
Rather than completely killing the kernel if we receive an esr value we
can't deal with in the el0 handlers, send the process a SIGILL and log
the esr value in the hope that we can debug it. If we receive a bad esr
from el1, we'll die() as before.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: stable@vger.kernel.org
Currently, if a compat process reads or writes from/to a disabled
cp15/cp14 register, the trap is not handled by the el0_sync_compat
handler, and the kernel will head to bad_mode, where it will die(), and
oops(). For 64 bit processes, disabled system register accesses are
currently treated as unhandled instructions.
This patch modifies entry.S to treat these unhandled traps as undefined
instructions, sending a SIGILL to userspace. This gives processes a
chance to handle this and stop using inaccessible registers, and
prevents further issues in the kernel as a result of the die().
Reported-by: Johannes Jensen <Johannes.Jensen@arm.com>
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The only reason uaccess routines might sleep
is if they fault. Make this explicit.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1369577426-26721-2-git-send-email-mst@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Currently user faults (page, undefined instruction) are always reported
even though the user may have a signal handler for them. This patch adds
unhandled_signal() check together with printk_ratelimit() for these
cases.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
When we take an exception at EL1, we only want to enable debug
exceptions if we're not currently stepping, otherwise we can easily get
stuck in a loop stepping into interrupt handlers.
Unfortunately, the current code tests the wrong bit in the mdscr, so fix
that.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
As per commit 764e0da1 (timers: Fixup the Kconfig consolidation
fallout), init/Kconfig already includes kernel/time/Kconfig, so no need
to do it explicitly for arm64.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The format of the lower 32-bits of the 64-bit operand to 'dc cisw' is
unchanged from ARMv7 architecture and the upper bits are RES0. This
implies that the 'way' field of the operand of 'dc cisw' occupies the
bit-positions [31 .. (32-A)]. Due to the use of 64-bit extended operands
to 'clz', the existing implementation of __flush_dcache_all is incorrectly
placing the 'way' field in the bit-positions [63 .. (64-A)].
Signed-off-by: Sukanto Ghosh <sghosh@apm.com>
Tested-by: Anup Patel <anup.patel@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: stable@vger.kernel.org
The of_platform_populate() is currently invoked at device_initcall()
level. There are however drivers that use platform_driver_probe()
directly and they need the devices to be populated. This patch makes the
of_platform_populate() and arch_initcall().
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Benoit Lecardonnel <Benoit.Lecardonnel@synopsys.com>
Tested-by: Benoit Lecardonnel <Benoit.Lecardonnel@synopsys.com>
