mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
synced 2025-01-15 11:57:46 +00:00
kheaders: prevent find
from seeing perl temp files
Symptom: The command find ... | xargs ... perl -i occasionally triggers error messages like the following, with the build still succeeding: Can't open <redacted>/kernel/.tmp_dir/include/dt-bindings/clock/XXNX4nW9: No such file or directory. Analysis: With strace, the root cause has been identified to be `perl -i` creating temporary files inside ${tmpdir}, which causes `find` to see the temporary files and emit the names. `find` is likely implemented with readdir. POSIX `readdir` says: If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. So if the libc that `find` links against choose to return that entry in readdir(), a possible sequence of events is the following: 1. find emits foo.h 2. xargs executes `perl -i foo.h` 3. perl (pid=100) creates temporary file `XXXXXXXX` 4. find sees file `XXXXXXXX` and emit it 5. PID 100 exits, cleaning up the temporary file `XXXXXXXX` 6. xargs executes `perl -i XXXXXXXX` 7. perl (pid=200) tries to read the file, but it doesn't exist any more. ... triggering the error message. One can reproduce the bug with the following command (assuming PWD contains the list of headers in kheaders.tar.xz) for i in $(seq 100); do find -type f -print0 | xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'; done With a `find` linking against musl libc, the error message is emitted 6/100 times. The fix: This change stores the results of `find` before feeding them into xargs. find and xargs will no longer be able to see temporary files that perl creates after this change. Signed-off-by: HONG Yifan <elsk@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
This commit is contained in:
parent
82a1978d0f
commit
41a0005128
@ -79,8 +79,13 @@ done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
|
|||||||
rm -f "${tmpdir}/include/generated/utsversion.h"
|
rm -f "${tmpdir}/include/generated/utsversion.h"
|
||||||
|
|
||||||
# Remove comments except SDPX lines
|
# Remove comments except SDPX lines
|
||||||
find "${tmpdir}" -type f -print0 |
|
# Use a temporary file to store directory contents to prevent find/xargs from
|
||||||
xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'
|
# seeing temporary files created by perl.
|
||||||
|
find "${tmpdir}" -type f -print0 > "${tmpdir}.contents.txt"
|
||||||
|
xargs -0 -P8 -n1 \
|
||||||
|
perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;' \
|
||||||
|
< "${tmpdir}.contents.txt"
|
||||||
|
rm -f "${tmpdir}.contents.txt"
|
||||||
|
|
||||||
# Create archive and try to normalize metadata for reproducibility.
|
# Create archive and try to normalize metadata for reproducibility.
|
||||||
tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
|
tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
|
||||||
|
Loading…
x
Reference in New Issue
Block a user