riscv: __asm_copy_to-from_user: Fix: overrun copy

There were two causes for the overrun memory access.

The threshold size was too small.
The aligning dst require one SZREG and unrolling word copy requires
8*SZREG, total have to be at least 9*SZREG.

Inside the unrolling copy, the subtracting -(8*SZREG-1) would make
iteration happening one extra loop. Proper value is -(8*SZREG).

Signed-off-by: Akira Tsukamoto <akira.tsukamoto@gmail.com>
Fixes: ca6eaaa210de ("riscv: __asm_copy_to-from_user: Optimize unaligned memory access and pipeline stall")
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
This commit is contained in:
Akira Tsukamoto 2021-07-20 17:50:52 +09:00 committed by Palmer Dabbelt
parent 76f5dfacfb
commit 6010d300f9
No known key found for this signature in database
GPG Key ID: 2E1319F35FBB1889

View File

@ -35,7 +35,7 @@ ENTRY(__asm_copy_from_user)
/* /*
* Use byte copy only if too small. * Use byte copy only if too small.
*/ */
li a3, 8*SZREG /* size must be larger than size in word_copy */ li a3, 9*SZREG /* size must be larger than size in word_copy */
bltu a2, a3, .Lbyte_copy_tail bltu a2, a3, .Lbyte_copy_tail
/* /*
@ -75,7 +75,7 @@ ENTRY(__asm_copy_from_user)
* a3 - a1 & mask:(SZREG-1) * a3 - a1 & mask:(SZREG-1)
* t0 - end of aligned dst * t0 - end of aligned dst
*/ */
addi t0, t0, -(8*SZREG-1) /* not to over run */ addi t0, t0, -(8*SZREG) /* not to over run */
2: 2:
fixup REG_L a4, 0(a1), 10f fixup REG_L a4, 0(a1), 10f
fixup REG_L a5, SZREG(a1), 10f fixup REG_L a5, SZREG(a1), 10f
@ -97,7 +97,7 @@ ENTRY(__asm_copy_from_user)
addi a1, a1, 8*SZREG addi a1, a1, 8*SZREG
bltu a0, t0, 2b bltu a0, t0, 2b
addi t0, t0, 8*SZREG-1 /* revert to original value */ addi t0, t0, 8*SZREG /* revert to original value */
j .Lbyte_copy_tail j .Lbyte_copy_tail
.Lshift_copy: .Lshift_copy: