Kernel/linux
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2024-12-28 16:53:49 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
fd0584d220
linux/scripts/Makefile.btf

36 lines
1.1 KiB
Makefile
Raw Normal View History

kbuild: avoid too many execution of scripts/pahole-flags.sh scripts/pahole-flags.sh is executed so many times. You can confirm it, as follows: $ cat <<EOF >> scripts/pahole-flags.sh > echo "scripts/pahole-flags.sh was executed" >&2 > EOF $ make -s scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed [ lots of repeated lines... ] This scripts is executed more than 20 times during the kernel build because PAHOLE_FLAGS is a recursively expanded variable and exported to sub-processes. With GNU Make >= 4.4, it is executed more than 60 times because exported variables are also passed to other $(shell ) invocations. Without careful coding, it is known to cause an exponential fork explosion. [1] The use of $(shell ) in an exported recursive variable is likely wrong because $(shell ) is always evaluated due to the 'export' keyword, and the evaluation can occur multiple times by the nature of recursive variables. Convert the shell script to a Makefile, which is included only when CONFIG_DEBUG_INFO_BTF=y. [1]: https://savannah.gnu.org/bugs/index.php?64746 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Nicolas Schier <n.schier@avm.de> Tested-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
2023-10-18 15:19:48 +00:00
# SPDX-License-Identifier: GPL-2.0
pahole-ver := $(CONFIG_PAHOLE_VERSION)
pahole-flags-y :=
kbuild,bpf: Pass make jobs' value to pahole Pass the value of make's -j/--jobs argument to pahole, to avoid out of memory errors and make pahole respect the "jobs" value of make. On systems with little memory but many cores, invoking pahole using -j without argument potentially creates too many pahole instances, causing an out-of-memory situation. Instead, we should pass make's "jobs" value as an argument to pahole's -j, which is likely configured to be (much) lower than the actual core count on such systems. If make was invoked without -j, either via cmdline or MAKEFLAGS, then JOBS will be simply empty, resulting in the existing behavior, as expected. Signed-off-by: Florian Schmaus <flo@geekplace.eu> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Link: https://lore.kernel.org/bpf/20241102100452.793970-1-flo@geekplace.eu
2024-11-02 10:04:51 +00:00
JOBS := $(patsubst -j%,%,$(filter -j%,$(MAKEFLAGS)))
kbuild,bpf: Switch to using --btf_features for pahole v1.26 and later The btf_features list can be used for pahole v1.26 and later - it is useful because if a feature is not yet implemented it will not exit with a failure message. This will allow us to add feature requests to the pahole options without having to check pahole versions in future; if the version of pahole supports the feature it will be added. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240507135514.490467-1-alan.maguire@oracle.com
2024-05-07 13:55:14 +00:00
ifeq ($(call test-le, $(pahole-ver), 125),y)
kbuild: avoid too many execution of scripts/pahole-flags.sh scripts/pahole-flags.sh is executed so many times. You can confirm it, as follows: $ cat <<EOF >> scripts/pahole-flags.sh > echo "scripts/pahole-flags.sh was executed" >&2 > EOF $ make -s scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed [ lots of repeated lines... ] This scripts is executed more than 20 times during the kernel build because PAHOLE_FLAGS is a recursively expanded variable and exported to sub-processes. With GNU Make >= 4.4, it is executed more than 60 times because exported variables are also passed to other $(shell ) invocations. Without careful coding, it is known to cause an exponential fork explosion. [1] The use of $(shell ) in an exported recursive variable is likely wrong because $(shell ) is always evaluated due to the 'export' keyword, and the evaluation can occur multiple times by the nature of recursive variables. Convert the shell script to a Makefile, which is included only when CONFIG_DEBUG_INFO_BTF=y. [1]: https://savannah.gnu.org/bugs/index.php?64746 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Nicolas Schier <n.schier@avm.de> Tested-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
2023-10-18 15:19:48 +00:00
# pahole 1.18 through 1.21 can't handle zero-sized per-CPU vars
ifeq ($(call test-le, $(pahole-ver), 121),y)
pahole-flags-$(call test-ge, $(pahole-ver), 118) += --skip_encoding_btf_vars
endif
pahole-flags-$(call test-ge, $(pahole-ver), 121) += --btf_gen_floats
kbuild,bpf: Pass make jobs' value to pahole Pass the value of make's -j/--jobs argument to pahole, to avoid out of memory errors and make pahole respect the "jobs" value of make. On systems with little memory but many cores, invoking pahole using -j without argument potentially creates too many pahole instances, causing an out-of-memory situation. Instead, we should pass make's "jobs" value as an argument to pahole's -j, which is likely configured to be (much) lower than the actual core count on such systems. If make was invoked without -j, either via cmdline or MAKEFLAGS, then JOBS will be simply empty, resulting in the existing behavior, as expected. Signed-off-by: Florian Schmaus <flo@geekplace.eu> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Link: https://lore.kernel.org/bpf/20241102100452.793970-1-flo@geekplace.eu
2024-11-02 10:04:51 +00:00
pahole-flags-$(call test-ge, $(pahole-ver), 122) += -j$(JOBS)
kbuild: avoid too many execution of scripts/pahole-flags.sh scripts/pahole-flags.sh is executed so many times. You can confirm it, as follows: $ cat <<EOF >> scripts/pahole-flags.sh > echo "scripts/pahole-flags.sh was executed" >&2 > EOF $ make -s scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed [ lots of repeated lines... ] This scripts is executed more than 20 times during the kernel build because PAHOLE_FLAGS is a recursively expanded variable and exported to sub-processes. With GNU Make >= 4.4, it is executed more than 60 times because exported variables are also passed to other $(shell ) invocations. Without careful coding, it is known to cause an exponential fork explosion. [1] The use of $(shell ) in an exported recursive variable is likely wrong because $(shell ) is always evaluated due to the 'export' keyword, and the evaluation can occur multiple times by the nature of recursive variables. Convert the shell script to a Makefile, which is included only when CONFIG_DEBUG_INFO_BTF=y. [1]: https://savannah.gnu.org/bugs/index.php?64746 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Nicolas Schier <n.schier@avm.de> Tested-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
2023-10-18 15:19:48 +00:00
kbuild, bpf: Use test-ge check for v1.25-only pahole There is no need to set the pahole v1.25-only flags in an "ifeq" version clause; we are already in a <= v1.25 branch of "ifeq", so that combined with a "test-ge" v1.25 ensures the flags will be applied for v1.25 only. Suggested-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240514162716.2448265-1-alan.maguire@oracle.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2024-05-14 16:27:16 +00:00
pahole-flags-$(call test-ge, $(pahole-ver), 125) += --skip_encoding_btf_inconsistent_proto --btf_gen_optimized
kbuild,bpf: Switch to using --btf_features for pahole v1.26 and later The btf_features list can be used for pahole v1.26 and later - it is useful because if a feature is not yet implemented it will not exit with a failure message. This will allow us to add feature requests to the pahole options without having to check pahole versions in future; if the version of pahole supports the feature it will be added. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240507135514.490467-1-alan.maguire@oracle.com
2024-05-07 13:55:14 +00:00
else
kbuild: avoid too many execution of scripts/pahole-flags.sh scripts/pahole-flags.sh is executed so many times. You can confirm it, as follows: $ cat <<EOF >> scripts/pahole-flags.sh > echo "scripts/pahole-flags.sh was executed" >&2 > EOF $ make -s scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed [ lots of repeated lines... ] This scripts is executed more than 20 times during the kernel build because PAHOLE_FLAGS is a recursively expanded variable and exported to sub-processes. With GNU Make >= 4.4, it is executed more than 60 times because exported variables are also passed to other $(shell ) invocations. Without careful coding, it is known to cause an exponential fork explosion. [1] The use of $(shell ) in an exported recursive variable is likely wrong because $(shell ) is always evaluated due to the 'export' keyword, and the evaluation can occur multiple times by the nature of recursive variables. Convert the shell script to a Makefile, which is included only when CONFIG_DEBUG_INFO_BTF=y. [1]: https://savannah.gnu.org/bugs/index.php?64746 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Nicolas Schier <n.schier@avm.de> Tested-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
2023-10-18 15:19:48 +00:00
kbuild,bpf: Switch to using --btf_features for pahole v1.26 and later The btf_features list can be used for pahole v1.26 and later - it is useful because if a feature is not yet implemented it will not exit with a failure message. This will allow us to add feature requests to the pahole options without having to check pahole versions in future; if the version of pahole supports the feature it will be added. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240507135514.490467-1-alan.maguire@oracle.com
2024-05-07 13:55:14 +00:00
# Switch to using --btf_features for v1.26 and later.
kbuild,bpf: Pass make jobs' value to pahole Pass the value of make's -j/--jobs argument to pahole, to avoid out of memory errors and make pahole respect the "jobs" value of make. On systems with little memory but many cores, invoking pahole using -j without argument potentially creates too many pahole instances, causing an out-of-memory situation. Instead, we should pass make's "jobs" value as an argument to pahole's -j, which is likely configured to be (much) lower than the actual core count on such systems. If make was invoked without -j, either via cmdline or MAKEFLAGS, then JOBS will be simply empty, resulting in the existing behavior, as expected. Signed-off-by: Florian Schmaus <flo@geekplace.eu> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Link: https://lore.kernel.org/bpf/20241102100452.793970-1-flo@geekplace.eu
2024-11-02 10:04:51 +00:00
pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j$(JOBS) --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs
kbuild,bpf: Switch to using --btf_features for pahole v1.26 and later The btf_features list can be used for pahole v1.26 and later - it is useful because if a feature is not yet implemented it will not exit with a failure message. This will allow us to add feature requests to the pahole options without having to check pahole versions in future; if the version of pahole supports the feature it will be added. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240507135514.490467-1-alan.maguire@oracle.com
2024-05-07 13:55:14 +00:00
kbuild,bpf: Add module-specific pahole flags for distilled base BTF Support creation of module BTF along with distilled base BTF; the latter is stored in a .BTF.base ELF section and supplements split BTF references to base BTF with information about base types, allowing for later relocation of split BTF with a (possibly changed) base. resolve_btfids detects the presence of a .BTF.base section and will use it instead of the base BTF it is passed in BTF id resolution. Modules will be built with a distilled .BTF.base section for external module build, i.e. make -C. -M=path2/module ...while in-tree module build as part of a normal kernel build will not generate distilled base BTF; this is because in-tree modules change with the kernel and do not require BTF relocation for the running vmlinux. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Reviewed-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240620091733.1967885-6-alan.maguire@oracle.com
2024-06-20 09:17:32 +00:00
ifneq ($(KBUILD_EXTMOD),)
module-pahole-flags-$(call test-ge, $(pahole-ver), 126) += --btf_features=distilled_base
endif
kbuild,bpf: Switch to using --btf_features for pahole v1.26 and later The btf_features list can be used for pahole v1.26 and later - it is useful because if a feature is not yet implemented it will not exit with a failure message. This will allow us to add feature requests to the pahole options without having to check pahole versions in future; if the version of pahole supports the feature it will be added. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240507135514.490467-1-alan.maguire@oracle.com
2024-05-07 13:55:14 +00:00
endif
pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust
kbuild: avoid too many execution of scripts/pahole-flags.sh scripts/pahole-flags.sh is executed so many times. You can confirm it, as follows: $ cat <<EOF >> scripts/pahole-flags.sh > echo "scripts/pahole-flags.sh was executed" >&2 > EOF $ make -s scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed [ lots of repeated lines... ] This scripts is executed more than 20 times during the kernel build because PAHOLE_FLAGS is a recursively expanded variable and exported to sub-processes. With GNU Make >= 4.4, it is executed more than 60 times because exported variables are also passed to other $(shell ) invocations. Without careful coding, it is known to cause an exponential fork explosion. [1] The use of $(shell ) in an exported recursive variable is likely wrong because $(shell ) is always evaluated due to the 'export' keyword, and the evaluation can occur multiple times by the nature of recursive variables. Convert the shell script to a Makefile, which is included only when CONFIG_DEBUG_INFO_BTF=y. [1]: https://savannah.gnu.org/bugs/index.php?64746 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Nicolas Schier <n.schier@avm.de> Tested-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
2023-10-18 15:19:48 +00:00
export PAHOLE_FLAGS := $(pahole-flags-y)
kbuild,bpf: Add module-specific pahole flags for distilled base BTF Support creation of module BTF along with distilled base BTF; the latter is stored in a .BTF.base ELF section and supplements split BTF references to base BTF with information about base types, allowing for later relocation of split BTF with a (possibly changed) base. resolve_btfids detects the presence of a .BTF.base section and will use it instead of the base BTF it is passed in BTF id resolution. Modules will be built with a distilled .BTF.base section for external module build, i.e. make -C. -M=path2/module ...while in-tree module build as part of a normal kernel build will not generate distilled base BTF; this is because in-tree modules change with the kernel and do not require BTF relocation for the running vmlinux. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Reviewed-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240620091733.1967885-6-alan.maguire@oracle.com
2024-06-20 09:17:32 +00:00
export MODULE_PAHOLE_FLAGS := $(module-pahole-flags-y)
Reference in New Issue Copy Permalink