Now modpost reads symbol versions from .*.cmd files.
The merged *.symversions are no longer needed.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
as a placeholder.
Genksyms writes the version CRCs into the linker script, which will be
used for filling the __crc_* symbols. The linker script format depends
on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
to the reference of CRC.
It is time to get rid of this complexity.
Now that modpost parses text files (.*.cmd) to collect all the CRCs,
it can generate C code that will be linked to the vmlinux or modules.
Generate a new C file, .vmlinux.export.c, which contains the CRCs of
symbols exported by vmlinux. It is compiled and linked to vmlinux in
scripts/link-vmlinux.sh.
Put the CRCs of symbols exported by modules into the existing *.mod.c
files. No additional build step is needed for modules. As before,
*.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
No linker magic is used here. The new C implementation works in the
same way, whether CONFIG_RELOCATABLE is enabled or not.
CONFIG_MODULE_REL_CRCS is no longer needed.
Previously, Kbuild invoked additional $(LD) to update the CRCs in
objects, but this step is unneeded too.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
When CONFIG_MODVERSIONS=y, the output from genksyms is saved in
separate *.symversions files, and will be used much later when
CONFIG_LTO_CLANG=y because it is impossible to update LLVM bit code
here.
This approach is not robust because:
- *.symversions may or may not exist. If *.symversions does not
exist, we never know if it is missing for legitimate reason
(i.e. no EXPORT_SYMBOL) or something bad has happened (for
example, the user accidentally deleted it). Once it occurs,
it is not self-healing because *.symversions is generated
as a side effect.
- stale (i.e. invalid) *.symversions might be picked up if an
object is generated in a non-ordinary way, and corresponding
*.symversions (, which was generated by old builds) just happen
to exist.
A more robust approach is to save symbol versions in *.cmd files
because:
- *.cmd always exists (if the object is generated by if_changed
rule or friends). Even if the user accidentally deletes it,
it will be regenerated in the next build.
- *.cmd is always re-generated when the object is updated. This
avoid stale version information being picked up.
I will remove *.symversions later.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
ld and ar support @file, which command-line options are read from.
Now that *.mod lists the member objects in the correct order, without
duplication, it is ready to be passed to ld and ar.
By using the @file syntax, people will not be worried about the pitfall
described in the NOTE.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
The dependency
$(obj)/%.mod: $(obj)/%$(mod-prelink-ext).o
... exists because *.mod files previously contained undefined symbols,
which are computed from *.o files when CONFIG_TRIM_UNUSED_KSYMS=y.
Now that the undefined symbols are put into separate *.usyms files,
there is no reason to make *.mod depend on *.o files.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
It is allowed to add the same objects multiple times to obj-y / obj-m:
obj-y += foo.o foo.o foo.o
obj-m += bar.o bar.o bar.o
It is also allowed to add the same objects multiple times to a composite
module:
obj-m += foo.o
foo-y := foo1.o foo2.o foo2.o foo1.o
This flexibility is useful because the same object might be selected by
different CONFIG options, like this:
obj-m += foo.o
foo-y := foo1.o
foo-$(CONFIG_FOO_X) += foo2.o
foo-$(CONFIG_FOO_Y) += foo2.o
The duplicated objects are omitted at link time. It works naturally in
Makefiles because GNU Make removes duplication in $^ without changing
the order.
It is working well, almost...
A small flaw I notice is, *.mod contains duplication in such a case.
This is probably not a big deal. As far as I know, the only small
problem is scripts/mod/sumversion.c parses the same file multiple
times.
I am fixing this because I plan to reuse *.mod for other purposes,
where the duplication can be problematic.
The code change is quite simple. We already use awk to drop duplicated
lines in modules.order (see cmd_modules_order in the same file).
I copied the code, but changed RS to use spaces as record separators.
I also changed the file format to list one object per line.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The *.mod files have two lines; the first line lists the member objects
of the module, and the second line, if CONFIG_TRIM_UNUSED_KSYMS=y, lists
the undefined symbols.
Currently, we generate *.mod after constructing composite modules,
otherwise, we cannot compute the second line. No prerequisite is
required to print the first line.
They are orthogonal. Splitting them into separate commands will ease
further cleanups.
This commit splits the list of undefined symbols out to *.usyms files.
Previously, the list of undefined symbols ended up with a very long
line, but now it has one symbol per line.
Use sed like we did before commit 7d32358be8ac ("kbuild: avoid split
lines in .mod files").
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
The first command in cmd_mod is similar to the real-search macro.
Reuse it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Split the code into two macros, cmd_gen_symversions_S for running
genksyms, and cmd_modversions for running $(LD) to update the object
with CRCs.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
cmd_modversions_c implements two parts; run genksyms to calculate CRCs
of exported symbols, run $(LD) to update the object with the CRCs. The
latter is not executed for CONFIG_LTO_CLANG=y since the object is not
ELF but LLVM bit code at this point.
The first part can be unified because we can always use $(NM) instead
of "$(OBJDUMP) -h" to dump the symbols.
Split the code into the two macros, cmd_gen_symversions_c and
cmd_modversions.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Presumably, 'test -s $@ || rm -f $@' intends to remove the output when
the genksyms command fails.
It is unneeded because .DELETE_ON_ERROR automatically removes the output
on failure.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
The genksyms command part in cmd_gensymtypes_{c,S} is duplicated.
Factor it out into the 'genksyms' macro.
For the readability, I slightly refactor the arguments to genksyms.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
- Add new environment variables, USERCFLAGS and USERLDFLAGS to allow
additional flags to be passed to user-space programs.
- Fix missing fflush() bugs in Kconfig and fixdep
- Fix a minor bug in the comment format of the .config file
- Make kallsyms ignore llvm's local labels, .L*
- Fix UAPI compile-test for cross-compiling with Clang
- Extend the LLVM= syntax to support LLVM=<suffix> form for using a
particular version of LLVm, and LLVM=<prefix> form for using custom
LLVM in a particular directory path.
- Clean up Makefiles
-----BEGIN PGP SIGNATURE-----
iQJJBAABCgAzFiEEbmPs18K1szRHjPqEPYsBB53g2wYFAmJFGloVHG1hc2FoaXJv
eUBrZXJuZWwub3JnAAoJED2LAQed4NsGH0kP/j6Vx5BqEv3tP2Q+UANxLqITleJs
IFpbSesz/BhlG7I/IapWmCDSqFbYd5uJTO4ko8CsPmZHcxr6Gw3y+DN5yQACKaG/
p9xiF6GjPyKR8+VdcT2tV50+dVY8ANe/DxCyzKrJd/uyYxgARPKJh0KRMNz+d9lj
ixUpCXDhx/XlKzPIlcxrvhhjevKz+NnHmN0fe6rzcOw9KzBGBTsf20Q3PqUuBOKa
rWHsRGcBPA8eKLfWT1Us1jjic6cT2g4aMpWjF20YgUWKHgWVKcNHpxYKGXASVo/z
ewdDnNfmwo7f7fKMCDDro9iwFWV/BumGtn43U00tnqdBcTpFojPlEOga37UPbZDF
nmTblGVUhR0vn4PmfBy8WkAkbW+IpVatKwJGV4J3KjSvdWvZOmVj9VUGLVAR0TXW
/YcgRs6EtG8Hn0IlCj0fvZ5wRWoDLbP2DSZ67R/44EP0GaNQPwUe4FI1izEE4EYX
oVUAIxcKixWGj4RmdtmtMMdUcZzTpbgS9uloMUmS3u9LK0Ir/8tcWaf2zfMO6Jl2
p4Q31s1dUUKCnFnj0xDKRyKGUkxYebrHLfuBqi0RIc0xRpSlxoXe3Dynm9aHEQoD
ZSV0eouQJxnaxM1ck5Bu4AHLgEebHfEGjWVyUHno7jFU5EI9Wpbqpe4pCYEEDTm1
+LJMEpdZO0dFvpF+
=84rW
-----END PGP SIGNATURE-----
Merge tag 'kbuild-v5.18-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada:
- Add new environment variables, USERCFLAGS and USERLDFLAGS to allow
additional flags to be passed to user-space programs.
- Fix missing fflush() bugs in Kconfig and fixdep
- Fix a minor bug in the comment format of the .config file
- Make kallsyms ignore llvm's local labels, .L*
- Fix UAPI compile-test for cross-compiling with Clang
- Extend the LLVM= syntax to support LLVM=<suffix> form for using a
particular version of LLVm, and LLVM=<prefix> form for using custom
LLVM in a particular directory path.
- Clean up Makefiles
* tag 'kbuild-v5.18-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
kbuild: Make $(LLVM) more flexible
kbuild: add --target to correctly cross-compile UAPI headers with Clang
fixdep: use fflush() and ferror() to ensure successful write to files
arch: syscalls: simplify uapi/kapi directory creation
usr/include: replace extra-y with always-y
certs: simplify empty certs creation in certs/Makefile
certs: include certs/signing_key.x509 unconditionally
kallsyms: ignore all local labels prefixed by '.L'
kconfig: fix missing '# end of' for empty menu
kconfig: add fflush() before ferror() check
kbuild: replace $(if A,A,B) with $(or A,B)
kbuild: Add environment variables for userprogs flags
kbuild: unify cmd_copy and cmd_shipped
Objtool's --ibt option generates .ibt_endbr_seal which lists
superfluous ENDBR instructions. That is those instructions for which
the function is never indirectly called.
Overwrite these ENDBR instructions with a NOP4 such that these
function can never be indirect called, reducing the number of viable
ENDBR targets in the kernel.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20220308154319.822545231@infradead.org
$(or ...) is available since GNU Make 3.81, and useful to shorten the
code in some places.
Covert as follows:
$(if A,A,B) --> $(or A,B)
This patch also converts:
$(if A, A, B) --> $(or A, B)
Strictly speaking, the latter is not an equivalent conversion because
GNU Make keeps spaces after commas; if A is not empty, $(if A, A, B)
expands to " A", while $(or A, B) expands to "A".
Anyway, preceding spaces are not significant in the code hunks I touched.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
For CONFIG_LTO_CLANG=y, the objtool processing is not possible at the
compilation, hence postponed by the link time.
Reuse $(cmd_objtool) for CONFIG_LTO_CLANG=y by defining objtool-enabled
properly.
For CONFIG_LTO_CLANG=y:
objtool-enabled is off for %.o compilation
objtool-enabled is on for %.lto link
For CONFIG_LTO_CLANG=n:
objtool-enabled is on for %.o compilation
(but, it depends on OBJECT_FILE_NON_STANDARD)
Set part-of-module := y for %.lto.o to avoid repeating --module.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Redo commit 8852c5524029 ("kbuild: Fix objtool dependency for
'OBJECT_FILES_NON_STANDARD_<obj> := n'") to add the objtool
dependency in a cleaner way.
Using .SECONDEXPANSION ends up with unreadable code due to escaped
dollars. Also, it is not efficient because the second half of
Makefile.build is parsed twice every time.
Append the objtool dependency to the *.cmd files at the build time.
This is what fixdep and gen_ksymdeps.sh already do. So, following the
same pattern seems a natural solution.
This allows us to drop $$(objtool_dep) entirely.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
The OBJECT_FILES_NON_STANDARD check is quite long.
Factor it out into a new macro, objtool-enabled, to not repeat it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
objtool_dep includes include/config/{ORC_UNWINDER,STACK_VALIDATION}
so that all the objects are rebuilt when CONFIG_ORC_UNWINDER or
CONFIG_STACK_VALIDATION is toggled.
BTW, the correct option name is not CONFIG_ORC_UNWINDER, but
CONFIG_UNWINDER_ORC. Commit 11af847446ed ("x86/unwind: Rename
unwinder config options to 'CONFIG_UNWINDER_*'") missed to
adjust this part. So, this dependency has been broken for a
long time.
As you can see in 'objtool_args', there are more CONFIG options
that affect the objtool command line.
Adding more and more include/config/* is ugly and unmaintainable.
Another issue is that non-standard objects are needlessly rebuilt.
Objects specified as OBJECT_FILES_NON_STANDARD is not processed by
objtool, but they are rebuilt anyway when CONFIG_STACK_VALIDATION
is toggled. This is not a big deal, but better to fix.
A cleaner and more precise fix is to include the objtool command in
*.cmd files so any command change is naturally detected by if_change.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Rename __objtool_obj to objtool, and move it out of the
'ifndef CONFIG_LTO_CLANG' conditional, so it can be used for
cmd_cc_lto_link_modules as well.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Commit b1a1a1a09b46 ("kbuild: lto: postpone objtool") moved objtool_args
to Makefile.lib, so the arguments can be used in Makefile.modfinal as
well as Makefile.build.
With commit 850ded46c642 ("kbuild: Fix TRIM_UNUSED_KSYMS with
LTO_CLANG"), module LTO linking came back to scripts/Makefile.build
again.
So, there is no more reason to keep objtool_args in a separate file.
Get it back to the original place, close to the objtool command.
Remove the stale comment too.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
cmd_update_lto_symversions merges all the existing *.symversions, but
some of them might be stale.
If the last EXPORT_SYMBOL is removed from a C file, the *.symversions
file is not deleted or updated. It contains stale CRCs, but still they
will be used for linking the vmlinux or modules.
It is not a big deal when the EXPORT_SYMBOL is really removed. However,
when the EXPORT_SYMBOL is moved to another file, the same __crc_<symbol>
will appear twice in the merged *.symversions, possibly with different
CRCs if the function argument is changed at the same time. It would
confuse module versioning.
If no EXPORT_SYMBOL is found, let's remove *.symversions explicitly.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
This is not used anywhere because the short log is displayed when
it is used through a $(call cmd,...) invocation.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
With CONFIG_LTO_CLANG, we currently link modules into native
code just before modpost, which means with TRIM_UNUSED_KSYMS
enabled, we still look at the LLVM bitcode in the .o files when
generating the list of used symbols. As the bitcode doesn't
yet have calls to compiler intrinsics and llvm-nm doesn't see
function references that only exist in function-level inline
assembly, we currently need a whitelist for TRIM_UNUSED_KSYMS to
work with LTO.
This change moves module LTO linking to happen earlier, and
thus avoids the issue with LLVM bitcode and TRIM_UNUSED_KSYMS
entirely, allowing us to also drop the whitelist from
gen_autoksyms.sh.
Link: https://github.com/ClangBuiltLinux/linux/issues/1369
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Alexander Lobakin <alobakin@pm.me>
Tested-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
When building modules(CONFIG_...=m), I found some of module versions
are incorrect and set to 0.
This can be found in build log for first clean build which shows
WARNING: EXPORT symbol "XXXX" [drivers/XXX/XXX.ko] version generation failed,
symbol will not be versioned.
But in second build(incremental build), the WARNING disappeared and the
module version becomes valid CRC and make someone who want to change
modules without updating kernel image can't insert their modules.
The problematic code is
+ $(foreach n, $(filter-out FORCE,$^), \
+ $(if $(wildcard $(n).symversions), \
+ ; cat $(n).symversions >> $@.symversions))
For example:
rm -f fs/notify/built-in.a.symversions ; rm -f fs/notify/built-in.a; \
llvm-ar cDPrST fs/notify/built-in.a fs/notify/fsnotify.o \
fs/notify/notification.o fs/notify/group.o ...
`foreach n` shows nothing to `cat` into $(n).symversions because
`if $(wildcard $(n).symversions)` return nothing, but actually
they do exist during this line was executed.
-rw-r--r-- 1 root root 168580 Jun 13 19:10 fs/notify/fsnotify.o
-rw-r--r-- 1 root root 111 Jun 13 19:10 fs/notify/fsnotify.o.symversions
The reason is the $(n).symversions are generated at runtime, but
Makefile wildcard function expends and checks the file exist or not
during parsing the Makefile.
Thus fix this by use `test` shell command to check the file
existence in runtime.
Rebase from both:
1. [https://lore.kernel.org/lkml/20210616080252.32046-1-lecopzer.chen@mediatek.com/]
2. [https://lore.kernel.org/lkml/20210702032943.7865-1-lecopzer.chen@mediatek.com/]
Fixes: 38e891849003 ("kbuild: lto: fix module versioning")
Co-developed-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
- Increase the -falign-functions alignment for the debug option.
- Remove ugly libelf checks from the top Makefile.
- Make the silent build (-s) more silent.
- Re-compile the kernel if KBUILD_BUILD_TIMESTAMP is specified.
- Various script cleanups
-----BEGIN PGP SIGNATURE-----
iQJJBAABCgAzFiEEbmPs18K1szRHjPqEPYsBB53g2wYFAmDon90VHG1hc2FoaXJv
eUBrZXJuZWwub3JnAAoJED2LAQed4NsGWFUP/RGNwlGD/YV1xg0ZmM0/ynBzzOy2
3dcr3etJZpipQDeqnHy3jt0esgMVlbkTdrHvP+2hpNaeXFwjF1fDHjhur9m8ZkVD
efOA6nugOnNwhy2G3BvtCJv+Vhb+KZ0nNLB27z3Bl0LGP6LJdMRNAxFBJMv4k3aR
F3sABugwCpnT2/YtuprxRl2/3/CyLur5NjY24FD+ugON3JIWfl6ETbHeFmxr1JE4
mE+zaN5AwYuSuH9LpdRy85XVCcW/FFqP/DwOFllVvCCCNvvS0KWYSNHWfEsKdR75
hmAAaS/rpi2eaL0vp88sNhAtYnhMSf+uFu0fyfYeWZuJqMt4Xz5xZKAzDsifCdif
aQ6UEPDjiKABh9gpX26BMd2CXzkGR+L4qZ7iBPfO586Iy7opajrFX9kIj5U7ZtCl
wsPat/9+18xpVJOTe0sss3idId7Ft4cRoW5FQMEAW2EWJ9fXAG1yDxEREj1V5gFx
sMXtpmCoQag968qjfARvP08s3MB1P4Ij6tXcioGqHuEWeJLxOMK/KWyafQUg611d
0kSWNO0OMo+odBj6j/vM+MIIaPhgwtZnPgw2q4uHGMcemzQxaEvGW+G/5a5qEpTv
SKm8W24wXplNot4tuTGWq5/jANRJcMvVsyC48DYT81OZEOWrIc0kDV4v4qZToTxW
97jn1NKa2H6L0J1V
=Za8V
-----END PGP SIGNATURE-----
Merge tag 'kbuild-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada:
- Increase the -falign-functions alignment for the debug option.
- Remove ugly libelf checks from the top Makefile.
- Make the silent build (-s) more silent.
- Re-compile the kernel if KBUILD_BUILD_TIMESTAMP is specified.
- Various script cleanups
* tag 'kbuild-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (27 commits)
scripts: add generic syscallnr.sh
scripts: check duplicated syscall number in syscall table
sparc: syscalls: use pattern rules to generate syscall headers
parisc: syscalls: use pattern rules to generate syscall headers
nds32: add arch/nds32/boot/.gitignore
kbuild: mkcompile_h: consider timestamp if KBUILD_BUILD_TIMESTAMP is set
kbuild: modpost: Explicitly warn about unprototyped symbols
kbuild: remove trailing slashes from $(KBUILD_EXTMOD)
kconfig.h: explain IS_MODULE(), IS_ENABLED()
kconfig: constify long_opts
scripts/setlocalversion: simplify the short version part
scripts/setlocalversion: factor out 12-chars hash construction
scripts/setlocalversion: add more comments to -dirty flag detection
scripts/setlocalversion: remove workaround for old make-kpkg
scripts/setlocalversion: remove mercurial, svn and git-svn supports
kbuild: clean up ${quiet} checks in shell scripts
kbuild: sink stdout from cmd for silent build
init: use $(call cmd,) for generating include/generated/compile.h
kbuild: merge scripts/mkmakefile to top Makefile
sh: move core-y in arch/sh/Makefile to arch/sh/Kbuild
...
I do not see a good reason why only the libelf development package must
be so carefully checked.
Kbuild generally does not check host tools or libraries.
For example, x86_64 defconfig fails to build with no libssl development
package installed.
scripts/extract-cert.c:21:10: fatal error: openssl/bio.h: No such file or directory
21 | #include <openssl/bio.h>
| ^~~~~~~~~~~~~~~
To solve the build error, you need to install libssl-dev or openssl-devel
package, depending on your distribution.
'apt-file search', 'dnf provides', etc. is your frined to find a proper
package to install.
This commit removes all the libelf checks from the top Makefile.
If libelf is missing, objtool will fail to build in a similar pattern:
.../linux/tools/objtool/include/objtool/elf.h:10:10: fatal error: gelf.h: No such file or directory
10 | #include <gelf.h>
You need to install libelf-dev, libelf-devel, or elfutils-libelf-devel
to proceed.
Another remarkable change is, CONFIG_STACK_VALIDATION (without
CONFIG_UNWINDER_ORC) previously continued to build with a warning,
but now it will treat missing libelf as an error.
This is just a one-time installation, so it should not hurt to break
a build and make a user install the package.
BTW, the traditional way to handle such checks is autotool, but according
to [1], I do not expect the kernel build would have similar scripting
like './configure' does.
[1]: https://lore.kernel.org/lkml/CA+55aFzr2HTZVOuzpHYDwmtRJLsVzE-yqg2DHpHi_9ePsYp5ug@mail.gmail.com/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
"OBJECT_FILES_NON_STANDARD_vma.o := n" has a dependency bug. When
objtool source is updated, the affected object doesn't get re-analyzed
by objtool.
Peter's new variable-sized jump label feature relies on objtool
rewriting the object file. Otherwise the system can fail to boot. That
effectively upgrades this minor dependency issue to a major bug.
The problem is that variables in prerequisites are expanded early,
during the read-in phase. The '$(objtool_dep)' variable indirectly uses
'$@', which isn't yet available when the target prerequisites are
evaluated.
Use '.SECONDEXPANSION:' which causes '$(objtool_dep)' to be expanded in
a later phase, after the target-specific '$@' variable has been defined.
Fixes: b9ab5ebb14ec ("objtool: Add CONFIG_STACK_VALIDATION option")
Fixes: ab3257042c26 ("jump_label, x86: Allow short NOPs")
Reported-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Rename overlay-y to multi-dtb-y, which is a consistent name with
multi-obj-y. Also, use multi-search to avoid code duplication.
Introduce real-dtb-y, which is a consistent name with real-obj-y,
to contain primitive blobs compiled from *.dts. This is used to
calculate the list of *.dt.yaml files.
Set -@ to base DTB without using $(eval ).
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
I think multi-obj-* is clearer, and more consistent with real-obj-*.
Rename as follows:
multi-used-y -> multi-obj-y
multi-used-m -> multi-obj-m
multi-used -> multi-obj-ym
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
scripts/Kbuild.include is included everywhere, but macros such as
cc-option are needed by build targets only.
For example, when 'make clean' traverses the tree, it does not need
to evaluate $(call cc-option,).
Split cc-option, ld-option, etc. to scripts/Makefile.compiler, which
is only included from the top Makefile and scripts/Makefile.build.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
VPATH is used in Kbuild to make pattern rules search for prerequisites
in both $(objtree) and $(srctree). Some of *.c, *.S files are not real
sources, but generated by tools such as flex, bison, perl.
In contrast, I doubt the benefit of --include-dir=$(abs_srctree) because
it is always clear which Makefiles are real sources, and which are not.
So, my hope is to add $(srctree)/ prefix to all check-in Makefiles,
then remove --include-dir=$(abs_srctree) flag in the future.
I am touching only some Kbuild core parts for now. Treewide fixes will
be needed to achieve this goal.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
- Fix false-positive build warnings for ARCH=ia64 builds
- Optimize dictionary size for module compression with xz
- Check the compiler and linker versions in Kconfig
- Fix misuse of extra-y
- Support DWARF v5 debug info
- Clamp SUBLEVEL to 255 because stable releases 4.4.x and 4.9.x
exceeded the limit
- Add generic syscall{tbl,hdr}.sh for cleanups across arches
- Minor cleanups of genksyms
- Minor cleanups of Kconfig
-----BEGIN PGP SIGNATURE-----
iQJJBAABCgAzFiEEbmPs18K1szRHjPqEPYsBB53g2wYFAmA3zhgVHG1hc2FoaXJv
eUBrZXJuZWwub3JnAAoJED2LAQed4NsG0C4P/A5hUNFdkYI+EffAWZiHn69t0S8j
M1GQkZildKu/yOfm6hp3mNwgHmYgw0aAuch1htkJuv+5rXRtoK77yw0xKbUqNHyO
VqkJWQPVUXJbWIDiu332NaETHbFTWCnPZKGmzcbVOBHbYsXUJPp17gROQ9ke0fQN
Ae6OV5WINhoS8UnjESWb3qOO87MdQTZ+9mP+NMnVh4kV1SUeMAXLFwFll66KZTkj
GXB330N3p9L0wQVljhXpQ/YPOd76wJNPhJWJ9+hKLFbWsedovzlHb+duprh1z1xe
7LLaq9dEbXxe1Uz0qmK76lupXxilYMyUupTW9HIYtIsY8br8DIoBOG0bn46LVnuL
/m+UQNfUFCYYePT7iZQNNc1DISQJrxme3bjq0PJzZTDukNnHJVahnj9x4RoNaF8j
Dc+JME0r2i8Ccp28vgmaRgzvSsb8Xtw5icwRdwzIpyt1ubs/+tkd/GSaGzQo30Q8
m8y1WOjovHNX7OGnOaOWBGoQAX/2k/VHeAediMsPqWUoOxwsLHYxG/4KtgwbJ5vc
gu/Fyk1GRDklZPpLdYFVvz8TGnqSDogJgF+7WolJ6YvPGAUIDAfd5Ky2sWayddlm
wchc3sKDVyh3lov23h0WQVTvLO9xl+NZ6THxoAGdYeQ0DUu5OxwH8qje/UpWuo1a
DchhNN+g5pa6n56Z
=sLxb
-----END PGP SIGNATURE-----
Merge tag 'kbuild-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada:
- Fix false-positive build warnings for ARCH=ia64 builds
- Optimize dictionary size for module compression with xz
- Check the compiler and linker versions in Kconfig
- Fix misuse of extra-y
- Support DWARF v5 debug info
- Clamp SUBLEVEL to 255 because stable releases 4.4.x and 4.9.x
exceeded the limit
- Add generic syscall{tbl,hdr}.sh for cleanups across arches
- Minor cleanups of genksyms
- Minor cleanups of Kconfig
* tag 'kbuild-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (38 commits)
initramfs: Remove redundant dependency of RD_ZSTD on BLK_DEV_INITRD
kbuild: remove deprecated 'always' and 'hostprogs-y/m'
kbuild: parse C= and M= before changing the working directory
kbuild: reuse this-makefile to define abs_srctree
kconfig: unify rule of config, menuconfig, nconfig, gconfig, xconfig
kconfig: omit --oldaskconfig option for 'make config'
kconfig: fix 'invalid option' for help option
kconfig: remove dead code in conf_askvalue()
kconfig: clean up nested if-conditionals in check_conf()
kconfig: Remove duplicate call to sym_get_string_value()
Makefile: Remove # characters from compiler string
Makefile: reuse CC_VERSION_TEXT
kbuild: check the minimum linker version in Kconfig
kbuild: remove ld-version macro
scripts: add generic syscallhdr.sh
scripts: add generic syscalltbl.sh
arch: syscalls: remove $(srctree)/ prefix from syscall tables
arch: syscalls: add missing FORCE and fix 'targets' to make if_changed work
gen_compile_commands: prune some directories
kbuild: simplify access to the kernel's version
...
These have no more user in the upstream code. The use of them has been
warned for a while for external modules. The migration is finished.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
With LTO, LLVM bitcode won't be compiled into native code until
modpost_link, or modfinal for modules. This change postpones calls
to objtool until after these steps, and moves objtool_args to
Makefile.lib, so the arguments can be reused in Makefile.modfinal.
As we didn't have objects to process earlier, we use --duplicate
when processing vmlinux.o. This change also disables unreachable
instruction warnings with LTO to avoid warnings about the int3
padding between functions.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
With CONFIG_MODVERSIONS, version information is linked into each
compilation unit that exports symbols. With LTO, we cannot use this
method as all C code is compiled into LLVM bitcode instead. This
change collects symbol versions into .symversions files and merges
them in link-vmlinux.sh where they are all linked into vmlinux.o at
the same time.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20201211184633.3213045-4-samitolvanen@google.com
This change adds build system support for Clang's Link Time
Optimization (LTO). With -flto, instead of ELF object files, Clang
produces LLVM bitcode, which is compiled into native code at link
time, allowing the final binary to be optimized globally. For more
details, see:
https://llvm.org/docs/LinkTimeOptimization.html
The Kconfig option CONFIG_LTO_CLANG is implemented as a choice,
which defaults to LTO being disabled. To use LTO, the architecture
must select ARCH_SUPPORTS_LTO_CLANG and support:
- compiling with Clang,
- compiling all assembly code with Clang's integrated assembler,
- and linking with LLD.
While using CONFIG_LTO_CLANG_FULL results in the best runtime
performance, the compilation is not scalable in time or
memory. CONFIG_LTO_CLANG_THIN enables ThinLTO, which allows
parallel optimization and faster incremental builds. ThinLTO is
used by default if the architecture also selects
ARCH_SUPPORTS_LTO_CLANG_THIN:
https://clang.llvm.org/docs/ThinLTO.html
To enable LTO, LLVM tools must be used to handle bitcode files, by
passing LLVM=1 and LLVM_IAS=1 options to make:
$ make LLVM=1 LLVM_IAS=1 defconfig
$ scripts/config -e LTO_CLANG_THIN
$ make LLVM=1 LLVM_IAS=1
To prepare for LTO support with other compilers, common parts are
gated behind the CONFIG_LTO option, and LTO can be disabled for
specific files by filtering out CC_FLAGS_LTO.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20201211184633.3213045-3-samitolvanen@google.com
Move function tracer options to Kconfig to make it easier to add
new methods for generating __mcount_loc, and to make the options
available also when building kernel modules.
Note that FTRACE_MCOUNT_USE_* options are updated on rebuild and
therefore, work even if the .config was generated in a different
environment.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20201211184633.3213045-2-samitolvanen@google.com
"xargs echo" is not a safe way to remove line breaks because the input
may exceed the command line limit and xargs may break it up into
multiple invocations of echo. This should never happen because
scripts/gen_autoksyms.sh expects all undefined symbols are placed in
the second line of .mod files.
One possible way is to replace "xargs echo" with
"sed ':x;N;$!bx;s/\n/ /g'" or something, but I rewrote the code by
using awk because it is more readable.
This issue was reported by Sami Tolvanen; in his Clang LTO patch set,
$(multi-used-m) is no longer an ELF object, but a thin archive that
contains LLVM bitcode files. llvm-nm prints out symbols for each
archive member separately, which results a lot of dupications, in some
places, beyond the system-defined limit.
This problem must be fixed irrespective of LTO, and we must ensure
zero possibility of having this issue.
Link: https://lkml.org/lkml/2020/12/1/1658
Reported-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
This change removes all instances of DISABLE_LTO from
Makefiles, as they are currently unused, and the preferred
method of disabling LTO is to filter out the flags instead.
Note added by Masahiro Yamada:
DISABLE_LTO was added as preparation for GCC LTO, but GCC LTO was
not pulled into the mainline. (https://lkml.org/lkml/2014/4/8/272)
Suggested-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The conditional:
ifneq ($(hostprogs),)
... is evaluated to true if $(hostprogs) does not contain any word but
whitespace characters.
ifneq ($(strip $(hostprogs)),)
... is a safe way to avoid interpreting whitespace as a non-empty value,
but I'd rather want to use the side-effect of $(sort ...) to do the
equivalent.
$(sort ...) is used in scripts/Makefile.host in order to drop duplication
in $(hostprogs). It is also useful to strip excessive spaces.
Move $(sort ...) before evaluating the ifneq.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The host shared library rules are currently implemented in
scripts/Makefile.host, but actually GCC-plugin is the only user of
them. (The VDSO .so files are built for the target by different
build rules) Hence, they do not need to be treewide available.
Move all the relevant build rules to scripts/gcc-plugins/Makefile.
I also optimized the build steps so *.so is directly built from .c
because every upstream plugin is compiled from a single source file.
I am still keeping the multi-file plugin support, which Kees Cook
mentioned might be needed by out-of-tree plugins.
(https://lkml.org/lkml/2019/1/11/1107)
If the plugin, foo.so, is compiled from two files foo.c and foo2.c,
then you can do like follows:
foo-objs := foo.o foo2.o
Single-file plugins do not need the *-objs notation.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Currently, the directories of objects are automatically created
only for O= builds.
It should not hurt to cater to this for in-tree builds too.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Since the pre-git time the checker is run first, before the compiler.
But if the source file contains some syntax error, the warnings from
the compiler are more useful than those from sparse (and other
checker most probably too).
So move the 'check' command to run after the compiler.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Make modules.order depend on $(obj-m), and use if_changed to build it.
This will avoid unneeded update of modules.order, which will be useful
to optimize the modpost stage.
Currently, the second pass of modpost is always invoked. By checking the
timestamp of modules.order, we can avoid the unneeded modpost.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
