The pcpu setup when using the page allocator sets up a new vmalloc
mapping very early in the boot process, so early that it cannot use the
flush_cache_vmap() function which may depend on structures not yet
initialized (for example in riscv, we currently send an IPI to flush
other cpus TLB).
But on some architectures, we must call flush_cache_vmap(): for example,
in riscv, some uarchs can cache invalid TLB entries so we need to flush
the new established mapping to avoid taking an exception.
So fix this by introducing a new function flush_cache_vmap_early() which
is called right after setting the new page table entry and before
accessing this new mapping. This new function implements a local flush
tlb on riscv and is no-op for other architectures (same as today).
Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Dennis Zhou <dennis@kernel.org>
Move the default (no-op) implementation of flush_icache_pages() to
<linux/cacheflush.h> from <asm-generic/cacheflush.h>. Remove the
flush_icache_page() wrapper from each architecture into
<linux/cacheflush.h>.
Link: https://lkml.kernel.org/r/20230802151406.3735276-32-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Add PFN_PTE_SHIFT, update_mmu_cache_range() and flush_dcache_folio().
Change the PG_dcache_clean flag from being per-page to per-folio.
Link: https://lkml.kernel.org/r/20230802151406.3735276-12-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Guo Ren <guoren@kernel.org>
Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Let's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the
offset. This reduces the maximum swap space per file to 16 GiB (was 32
GiB).
We might actually be able to reuse one of the other software bits
(_PAGE_READ / PAGE_WRITE) instead, because we only have to keep
pte_present(), pte_none() and HW happy. For now, let's keep it simple
because there might be something non-obvious.
Link: https://lkml.kernel.org/r/20230113171026.582290-6-david@redhat.com
Signed-off-by: David Hildenbrand <david@redhat.com>
Cc: Guo Ren <guoren@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
: error: C++ style comments are not allowed in ISO C90
// Copyright (C) 2018 Hangzhou C-SKY Microsystems co.,ltd.
^
error: (this will be reported only once per input file)
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
It could help to reduce the latency of the time-related functions
in user space.
We have referenced arm's and riscv's implementation for the patch.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Vincent Chen <vincent.chen@sifive.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Current csky's swappon is broken by wrong swap PTE entry format.
Now redesign the new format for abiv1 & abiv2 and make swappon +
zram work properly on csky machines.
C-SKY PTE has VALID, DIRTY to emulate PRESENT, READ, WRITE, EXEC
attributes. GLOBAL bit is shared by two pages in the same tlb
entry. So we need to keep GLOBAL, VALID, PRESENT zero in swp_pte.
To distinguish PAGE_NONE and swp_pte, we need to use an additional
bit (abiv1 is _PAGE_READ, abiv2 is _PAGE_WRITE).
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Reconstruct vdso framework to support future vsyscall,
vgettimeofday features. These are very important features to reduce
system calls into the kernel for performance improvement.
The patch is reference RISC-V's
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Palmer Dabbelt <palmerdabbelt@google.com>
TLB invalidate didn't contain a barrier operation in csky cpu and
we need to prevent previous PTW response after TLB invalidation
instruction. Of cause, the ASID changing also needs to take care
of the issue.
CPU0 CPU1
=============== ===============
set_pte
sync_is() -> See the previous set_pte for all harts
tlbi.vas -> Invalidate all harts TLB entry & flush pipeline
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
There are two ways for translating va to pa for csky:
- Use TLB(Translate Lookup Buffer) and PTW (Page Table Walk)
- Use SSEG0/1 (Simple Segment Mapping)
We use tlb mapping 0-2G and 3G-4G virtual address area and SSEG0/1
are for 2G-2.5G and 2.5G-3G translation. We could disable SSEG0
to use 2G-2.5G as TLB user mapping.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
There is no user return for RESTORE_REGS_FTRACE, so it's no need to
save sp into ss0 as RESTORE_REGS_ALL.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Arnd Bergmann <arnd@arndb.de>
There is no fixup or feature in the patch, we only cleanup with:
- Remove unnecessary reg used (r11, r12), just use r9 & r10 &
syscallid regs as temp useage.
- Add _TIF_SYSCALL_WORK and _TIF_WORK_MASK to gather macros.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Current implementation could destory a4 & a5 when strace, so we need to get them
from pt_regs by SAVE_ALL.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Just as comment mentioned, the msa format:
cr<30/31, 15> MSA register format:
31 - 29 | 28 - 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
BA Reserved SH WA B SO SEC C D V
So we should shift 29 bits not 28 bits for mask
Signed-off-by: Liu Yibin <jiulong@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
For the memory size ( > 512MB, < 1GB), the MSA setting is:
- SSEG0: PHY_START , PHY_START + 512MB
- SSEG1: PHY_START + 512MB, PHY_START + 1GB
But the real memory is no more than 1GB, there is a gap between the
end size of memory and border of 1GB. CPU could speculatively
execute to that gap and if the gap of the bus couldn't respond to
the CPU request, then the crash will happen.
Now make the setting with:
- SSEG0: PHY_START , PHY_START + 512MB (no change)
- SSEG1: Disabled (We use highmem to use the memory of 512MB~1GB)
We also deprecated zhole_szie[] settings, it's only used by arm
style CPUs. All memory gap should use Reserved setting of dts in
csky system.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
This patch implements FTRACE_WITH_REGS for csky, which allows a traced
function's arguments (and some other registers) to be captured into a
struct pt_regs, allowing these to be inspected and/or modified.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
WARNING: vmlinux.o(.text+0x2366): Section mismatch in reference from the
function csky_start_secondary() to the function .init.text:init_fpu()
The function csky_start_secondary() references
the function __init init_fpu().
This is often because csky_start_secondary lacks a __init
annotation or the annotation of init_fpu is wrong.
Reported-by: Lu Chongzhi <chongzhi.lcz@alibaba-inc.com>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Some CPUs don't support icache.va instruction to maintain the whole
smp cores' icache. Using icache.all + IPI casue a lot on performace
and using defer mechanism could reduce the number of calling icache
_flush_all functions.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Only when vma is for VM_EXEC, we need sync dcache & icache. eg:
- gdb ptrace modify user space instruction code area.
Add VM_EXEC condition to reduce unnecessary cache flush.
The abiv1 cpus' cache are all VIPT, so we still need to deal with
dcache aliasing problem. But there is optimized way to use cache
color, just like what's done in arch/csky/abiv1/inc/abi/page.h.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Instead of flushing cache per update_mmu_cache() called, we use
flush_dcache_page to reduce the frequency of flashing the cache.
As abiv2 cpus are all PIPT for icache & dcache, we needn't handle
dcache aliasing problem. But their icache can't snoop dcache, so
we still need sync_icache_dcache in update_mmu_cache().
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
The abiv2 CPUs are all PIPT cache, so there is no need to implement
flush_icache_page function.
The function flush_icache_user_range hasn't been used, so just
remove it.
The function flush_cache_range is not necessary for PIPT cache when
tlb mapping changed.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
In the past, we didn't care about kernel sp when saving pt_reg. But in some
cases, we still need pt_reg->usp to represent the kernel stack before enter
exception.
For cmpxhg in atomic.S, we need save and restore usp for above.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Use linux generic asid/vmid algorithm to implement csky
switch_mm function. The algorithm is from arm and it could
work with SMP system. It'll help reduce tlb flush for
switch_mm in task/vm switch.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
The name of phys_offset is so common for global export and it may
conflict with some local name. So change phys_offset to va_pa_offset
which also used by riscv.
Also use __pa() and __va() instead of using phys_offset directly.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Modify SETUP_MMU macro to fit on both MMU-on or MMU-off enviornment
and vmlinux could bootup from MMU off enviornment for some cases.
Unify the style of _start and _start_smp_secondary in head.S to make
head.S looks more concise and easy to understand.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
In trace events as tracepoints context are not able to
be retrieve with task_pt_regs. Without arch caller regs
support the pt_regs context will be all zero, perf can
not parsing the callchain and resolving the symbols
correctly, some time will even get into deadlock
while handling the page fault, eg:
perf kmem —page record ls
Changelog
- Add test case cmd in comment
- Use regs_fp(regs) which is defined in abi/regdef.h
Signed-off-by: Mao Han <han_mao@c-sky.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
Before this patch csky-linux need CONFIG_RAM_BASE to determine start
physical address. Now we use phys_offset variable to replace the macro
of PHYS_OFFSET and we setup phys_offset with real physical address which
is determined during startup in head.S.
With this patch we needn't re-compile kernel for different start
physical address. ie: 0x0 / 0xc0000000 start physical address could use
the same vmlinux, be care different start address must be 512MB aligned.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Linux kernel has provided some apis for arch signal's implementation.
For example:
restore_saved_sigmask()
set_current_blocked()
restore_altstack()
But in last version of csky signal.c didn't use them and some codes are
confusing, so reconstruct signal.c with reference to riscv's code.
Now csky signal.c implementation are very close to riscv and we can
get the following benefits:
- Clear code structure
- The signal code of riscv and csky can be reviewed together
- Promoting the unification of arch's signal implementation
Also modified the related code in entry.S
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
We could use regs->sr 16-24 bits to detect syscall: VEC_TRAP0 and
r11_sig is no necessary for current implementation.
In this patch, we implement the in_syscall and forget_syscall which are
inspired from arm & nds32, but csky pt_regs has no syscall_num element
and we just set zero to regs->sr's vector-bits-field instead.
For ret_from_fork, current task was forked from parent which is in syscall
progress and its regs->sr has been already setted with VEC_TRAP0. See:
arch/csky/kernel/process.c: copy_thread()
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Previous syscall_trace implementation couldn't support AUDITSYSCALL and
SYSCALL_TRACEPOINTS. Now we redesign it to support audit_syscall
and syscall_tracepoints just like other archs'.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Dmitry V. Levin <ldv@altlinux.org>
Cc: Arnd Bergmann <arnd@arndb.de>
HI, LO, DSPCR registers are 807/810 related regs and no need for 610/860.
All of the regs must be saved in pt_regs and switch_stack. This patch
fixup saving dspcr reg in switch_stack and pt_regs.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Glibc function mmap(... O_SYNC) will make page to _PAGE_UNCACHE +
_PAGE_SO and strong-order page couldn't support unalignment access.
So remove _PAGE_SO from _PAGE_UNCACHE, also sync abiv1 with the macro
of _PAGE_SO.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Reported-by: Liu Renwei <Renwei.Liu@verisilicon.com>
Tested-by: Yuan Qiyun <qiyun_yuan@c-sky.com>
This patch adds files related to VDSO and our VDSO only support
rt_sigreturn.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
This patch adds files related to task_switch, sigcontext, signal,
fpu context switch.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Eric W. Biederman <ebiederm@xmission.com>
This patch adds files related to memory management and here is our
memory-layout:
Fixmap : 0xffc02000 – 0xfffff000 (4 MB - 12KB)
Pkmap : 0xff800000 – 0xffc00000 (4 MB)
Vmalloc : 0xf0200000 – 0xff000000 (238 MB)
Lowmem : 0x80000000 – 0xc0000000 (1GB)
abiv1 CPU (CK610) is VIPT cache and it doesn't support highmem.
abiv2 CPUs are all PIPT cache and they could support highmem.
Lowmem is directly mapped by msa0 & msa1 reg, and we needn't setup
memory page table for it.
Link:https://lore.kernel.org/lkml/20180518215548.GH17671@n2100.armlinux.org.uk/
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Christoph Hellwig <hch@infradead.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
