mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2025-01-06 05:13:18 +00:00
347aa8dec2
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
WANG Rui
|
8941e93ca5 |
LoongArch: Optimize memory ops (memset/memcpy/memmove)
To optimize memset()/memcpy()/memmove() and so on, we use a jump table to dispatch cases for short data lengths; and for long data lengths, we split the destination into head part (first 8 bytes), tail part (last 8 bytes) and middle part. The head part and tail part may be at unaligned addresses, while the middle part is always aligned (the middle part is allowed to overlap the head/tail part). In this way, the first and last 8 bytes may be unaligned accesses, but we can make sure the data in the middle is processed at an aligned destination address. We have tested micro-bench[1] on a Loongson-3C5000 16-core machine (2.2GHz): 1. memset | length | src offset | dst offset | speed before | speed after | % | |--------|------------|------------|--------------|-------------|---------| | 8 | 0 | 0 | 696.191 | 1518.785 | 118.16% | | 8 | 0 | 1 | 696.325 | 1518.937 | 118.14% | | 50 | 0 | 0 | 969.976 | 8053.902 | 730.32% | | 50 | 0 | 1 | 970.034 | 8058.475 | 730.74% | | 300 | 0 | 0 | 5876.612 | 16544.703 | 181.53% | | 300 | 0 | 1 | 5030.849 | 16549.011 | 228.95% | | 1200 | 0 | 0 | 11797.077 | 16752.137 | 42.00% | | 1200 | 0 | 1 | 5687.141 | 16645.233 | 192.68% | | 4000 | 0 | 0 | 15723.27 | 16761.557 | 6.60% | | 4000 | 0 | 1 | 5906.114 | 16732.316 | 183.30% | | 8000 | 0 | 0 | 16751.403 | 16770.002 | 0.11% | | 8000 | 0 | 1 | 5995.449 | 16754.07 | 179.45% | 2. memcpy | length | src offset | dst offset | speed before | speed after | % | |--------|------------|------------|--------------|-------------|---------| | 8 | 0 | 0 | 696.2 | 1670.605 | 139.96% | | 8 | 0 | 1 | 696.325 | 1671.138 | 139.99% | | 50 | 0 | 0 | 969.974 | 8724.999 | 799.51% | | 50 | 0 | 1 | 970.032 | 8730.138 | 799.98% | | 300 | 0 | 0 | 5564.662 | 16272.652 | 192.43% | | 300 | 0 | 1 | 4670.436 | 14972.842 | 220.59% | | 1200 | 0 | 0 | 10740.23 | 16751.728 | 55.97% | | 1200 | 0 | 1 | 5027.741 | 14874.564 | 195.85% | | 4000 | 0 | 0 | 15122.367 | 16737.642 | 10.68% | | 4000 | 0 | 1 | 5536.918 | 14890.397 | 168.93% | | 8000 | 0 | 0 | 16505.453 | 16553.543 | 0.29% | | 8000 | 0 | 1 | 5821.619 | 14841.804 | 154.94% | 3. memmove | length | src offset | dst offset | speed before | speed after | % | |--------|------------|------------|--------------|-------------|---------| | 8 | 0 | 0 | 982.693 | 1670.568 | 70.00% | | 8 | 0 | 1 | 983.023 | 1671.174 | 70.00% | | 50 | 0 | 0 | 1230.87 | 8727.625 | 609.06% | | 50 | 0 | 1 | 1232.515 | 8730.138 | 608.32% | | 300 | 0 | 0 | 6490.375 | 16296.993 | 151.09% | | 300 | 0 | 1 | 4282.687 | 14972.842 | 249.61% | | 1200 | 0 | 0 | 11742.755 | 16752.546 | 42.66% | | 1200 | 0 | 1 | 5039.338 | 14872.951 | 195.14% | | 4000 | 0 | 0 | 15467.786 | 16737.09 | 8.21% | | 4000 | 0 | 1 | 5009.905 | 14890.542 | 197.22% | | 8000 | 0 | 0 | 16489.664 | 16553.273 | 0.39% | | 8000 | 0 | 1 | 5823.786 | 14858.646 | 155.14% | * speed: MB/s * length: byte [1] https://github.com/heiher/mem-bench Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn> |
||
|
Tiezhu Yang
|
fcf77d0162 |
LoongArch: Mark some assembler symbols as non-kprobe-able
Some assembler symbols are not kprobe safe, such as handle_syscall (used as syscall exception handler), *memset*/*memcpy*/*memmove* (may cause recursive exceptions), they can not be instrumented, just blacklist them for kprobing. Here is a related problem and discussion: Link: https://lore.kernel.org/lkml/20230114143859.7ccc45c1c5d9ce302113ab0a@kernel.org/ Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn> |
||
|
Huacai Chen
|
a275a82dcd |
LoongArch: Use alternative to optimize libraries
Use the alternative to optimize common libraries according whether CPU
has UAL (hardware unaligned access support) feature, including memset(),
memcopy(), memmove(), copy_user() and clear_user().
We have tested UnixBench on a Loongson-3A5000 quad-core machine (1.6GHz):
1, One copy, before patch:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 9566582.0 819.8
Double-Precision Whetstone 55.0 2805.3 510.1
Execl Throughput 43.0 2120.0 493.0
File Copy 1024 bufsize 2000 maxblocks 3960.0 209833.0 529.9
File Copy 256 bufsize 500 maxblocks 1655.0 89400.0 540.2
File Copy 4096 bufsize 8000 maxblocks 5800.0 320036.0 551.8
Pipe Throughput 12440.0 340624.0 273.8
Pipe-based Context Switching 4000.0 109939.1 274.8
Process Creation 126.0 4728.7 375.3
Shell Scripts (1 concurrent) 42.4 2223.1 524.3
Shell Scripts (8 concurrent) 6.0 883.1 1471.9
System Call Overhead 15000.0 518639.1 345.8
========
System Benchmarks Index Score 500.2
2, One copy, after patch:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 9567674.7 819.9
Double-Precision Whetstone 55.0 2805.5 510.1
Execl Throughput 43.0 2392.7 556.4
File Copy 1024 bufsize 2000 maxblocks 3960.0 417804.0 1055.1
File Copy 256 bufsize 500 maxblocks 1655.0 112909.5 682.2
File Copy 4096 bufsize 8000 maxblocks 5800.0 1255207.4 2164.2
Pipe Throughput 12440.0 555712.0 446.7
Pipe-based Context Switching 4000.0 99964.5 249.9
Process Creation 126.0 5192.5 412.1
Shell Scripts (1 concurrent) 42.4 2302.4 543.0
Shell Scripts (8 concurrent) 6.0 919.6 1532.6
System Call Overhead 15000.0 511159.3 340.8
========
System Benchmarks Index Score 640.1
3, Four copies, before patch:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 38268610.5 3279.2
Double-Precision Whetstone 55.0 11222.2 2040.4
Execl Throughput 43.0 7892.0 1835.3
File Copy 1024 bufsize 2000 maxblocks 3960.0 235149.6 593.8
File Copy 256 bufsize 500 maxblocks 1655.0 74959.6 452.9
File Copy 4096 bufsize 8000 maxblocks 5800.0 545048.5 939.7
Pipe Throughput 12440.0 1337359.0 1075.0
Pipe-based Context Switching 4000.0 473663.9 1184.2
Process Creation 126.0 17491.2 1388.2
Shell Scripts (1 concurrent) 42.4 6865.7 1619.3
Shell Scripts (8 concurrent) 6.0 1015.9 1693.1
System Call Overhead 15000.0 1899535.2 1266.4
========
System Benchmarks Index Score 1278.3
4, Four copies, after patch:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 38272815.5 3279.6
Double-Precision Whetstone 55.0 11222.8 2040.5
Execl Throughput 43.0 8839.2 2055.6
File Copy 1024 bufsize 2000 maxblocks 3960.0 313912.9 792.7
File Copy 256 bufsize 500 maxblocks 1655.0 80976.1 489.3
File Copy 4096 bufsize 8000 maxblocks 5800.0 1176594.3 2028.6
Pipe Throughput 12440.0 2100941.9 1688.9
Pipe-based Context Switching 4000.0 476696.4 1191.7
Process Creation 126.0 18394.7 1459.9
Shell Scripts (1 concurrent) 42.4 7172.2 1691.6
Shell Scripts (8 concurrent) 6.0 1058.3 1763.9
System Call Overhead 15000.0 1874714.7 1249.8
========
System Benchmarks Index Score 1488.8
Signed-off-by: Jun Yi <yijun@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
|