When testing a `clang` upgrade with Rust Binder, Alice encountered [1] a
build failure caused by `bindgen` not translating some symbols related to
tracepoints. This was caused by commit 2e770edd8ce1 ("[libclang] Compute
the right spelling location") changing the behavior of a function exposed
by `libclang`. `bindgen` fixed the regression in commit 600f63895f73
("Use clang_getFileLocation instead of clang_getSpellingLocation").
However, the regression fix is only available in `bindgen` versions
0.69.5 or later (it was backported for 0.69.x). This means that when
older bindgen versions are used with new versions of `libclang`, `bindgen`
may do the wrong thing, which could lead to a build failure.
Alice encountered the bug with some header files related to tracepoints,
but it could also cause build failures in other circumstances. Thus,
always emit a warning when using an old `bindgen` with a new `libclang`
so that other people do not have to spend time chasing down the same
bug.
However, testing just the version is inconvenient, since distributions
do patch their packages without changing the version, so I reduced the
issue into the following piece of code that can trigger the issue:
#define F(x) int x##x
F(foo);
In particular, an unpatched `bindgen` will ignore the macro expansion
and thus not provide a declaration for the exported `int`.
Thus add a build test to `rust_is_available.sh` using the code above
(that is only triggered if the versions appear to be affected), following
what we did for the 0.66.x issue.
Moreover, I checked the status in the major distributions we have
instructions for:
- Fedora 41 was affected but is now OK, since it now ships `bindgen`
0.69.5.
Thanks Ben for the quick reply on the updates that were ongoing.
Fedora 40 and earlier are OK (older `libclang`, and they also now
carry `bindgen` 0.69.5).
- Debian Sid was affected but is now OK, since they now ship a patched
`bindgen` binary (0.66.1-7+b3). The issue was reported to Debian by
email and then as a bug report [2].
Thanks NoisyCoil and Matthias for the quick replies. NoisyCoil handled
the needed updates. Debian may upgrade to `bindgen` 0.70.x, too.
Debian Testing is OK (older `libclang` so far).
- Ubuntu non-LTS (oracular) is affected. The issue was reported to Ubuntu
by email and then as a bug report [3].
Ubuntu LTS is not affected (older `libclang` so far).
- Arch Linux, Gentoo Linux and openSUSE should be OK (newer `bindgen` is
provided). Nix as well (older `libclang` so far).
This issue was also added to our "live list" that tracks issues around
distributions [4].
Cc: Ben Beasley <code@musicinmybrain.net>
Cc: NoisyCoil <noisycoil@tutanota.com>
Cc: Matthias Geiger <werdahias@riseup.net>
Link: https://lore.kernel.org/rust-for-linux/20241030-bindgen-libclang-warn-v1-1-3a7ba9fedcfe@google.com/ [1]
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1086510 [2]
Link: https://bugs.launchpad.net/ubuntu/+source/rust-bindgen-cli/+bug/2086639 [3]
Link: https://github.com/Rust-for-Linux/linux/issues/1127 [4]
Co-developed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Link: https://lore.kernel.org/r/20241111201607.653149-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
`bindgen` versions 0.66.0 and 0.66.1 panic due to C string literals with
NUL characters [1]:
panicked at .cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.66.0/codegen/mod.rs:717:71:
called `Result::unwrap()` on an `Err` value: FromBytesWithNulError { kind: InteriorNul(4) }
Thus, in preparation for supporting several `bindgen` versions, add a
version check to warn the user about it.
Since some distributions may have patched it (e.g. Debian did [2]),
check if that seems to be the case (after the version check matches),
in order to avoid printing a warning in that case.
We could make it an error, but 1) it is going to fail anyway later
in the build, 2) we would disable `RUST`, which is also painful, 3)
someone could have patched it in a way that still makes our extra check
fail (however unlikely), 4) the interior NUL may go away in the headers
(however unlikely). Thus just warn about it so that users know why it
is failing.
In addition, add a couple tests for the new cases.
Link: https://github.com/rust-lang/rust-bindgen/pull/2567 [1]
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069047 [2]
Link: https://lore.kernel.org/r/20240709160615.998336-11-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
With both the workaround for `bindgen` 0.69.0 and the warning about
0.66.0 and 0.66.1 in place, start supporting several `bindgen` versions,
like it was done for the Rust compiler in a previous patch.
All other versions, including the latest 0.69.4, build without errors.
The `bindgen` project, like Rust, has also agreed to have the kernel
in their CI [1] -- thanks! This should help both projects: `bindgen`
will be able to detect early issues like those mentioned above, and the
kernel will be very likely build with new releases (at least for the
basic configuration being tested).
Link: https://github.com/rust-lang/rust-bindgen/pull/2851 [1]
Tested-by: Benno Lossin <benno.lossin@proton.me>
Tested-by: Andreas Hindborg <a.hindborg@samsung.com>
Link: https://lore.kernel.org/r/20240709160615.998336-10-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
`bindgen` 0.69.0 contains a bug: `--version` does not work without
providing a header [1]:
error: the following required arguments were not provided:
<HEADER>
Usage: bindgen <FLAGS> <OPTIONS> <HEADER> -- <CLANG_ARGS>...
Thus, in preparation for supporting several `bindgen` versions, work
around the issue by passing a dummy argument.
Include a comment so that we can remove the workaround in the future.
Link: https://github.com/rust-lang/rust-bindgen/pull/2678 [1]
Reviewed-by: Finn Behrens <me@kloenk.dev>
Tested-by: Benno Lossin <benno.lossin@proton.me>
Tested-by: Andreas Hindborg <a.hindborg@samsung.com>
Link: https://lore.kernel.org/r/20240709160615.998336-9-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
It is time to start supporting several Rust compiler versions and thus
establish a minimum Rust version.
We may still want to upgrade the minimum sometimes in the beginning since
there may be important features coming into the language that improve
how we write code (e.g. field projections), which may or may not make
sense to support conditionally.
We will start with a window of two stable releases, and widen it over
time. Thus this patch does not move the current minimum (1.78.0), but
instead adds support for the recently released 1.79.0.
This should already be enough for kernel developers in distributions that
provide recent Rust compiler versions routinely, such as Arch Linux,
Debian Unstable (outside the freeze period), Fedora Linux, Gentoo
Linux (especially the testing channel), Nix (unstable) and openSUSE
Tumbleweed. See the documentation patch about it later in this series.
In addition, Rust for Linux is now being built-tested in Rust's pre-merge
CI [1]. That is, every change that is attempting to land into the Rust
compiler is tested against the kernel, and it is merged only if it passes
-- thanks to the Rust project for that!
Thus, with the pre-merge CI in place, both projects hope to avoid
unintentional changes to Rust that break the kernel. This means that,
in general, apart from intentional changes on their side (that we will
need to workaround conditionally on our side), the upcoming Rust compiler
versions should generally work.
For instance, currently, the beta (1.80.0) and nightly (1.81.0) branches
work as well.
Of course, the Rust for Linux CI job in the Rust toolchain may still need
to be temporarily disabled for different reasons, but the intention is
to help bring Rust for Linux into stable Rust.
Link: https://github.com/rust-lang/rust/pull/125209 [1]
Reviewed-by: Finn Behrens <me@kloenk.dev>
Tested-by: Benno Lossin <benno.lossin@proton.me>
Tested-by: Andreas Hindborg <a.hindborg@samsung.com>
Link: https://lore.kernel.org/r/20240709160615.998336-7-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
The script already checks for `$RUSTC` and `$BINDGEN` existing
and exiting without failure. However, one may still pass an
unexpected binary that does not output what the later parsing
expects. The script still successfully reports a failure as
expected, but the error is confusing. For instance:
$ RUSTC=true BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 * + 100 * + "
***
*** Please see Documentation/rust/quick-start.rst for details
*** on how to set up the Rust support.
***
Thus add an explicit check and a proper message for unexpected
output from the called command.
Similarly, do so for the `libclang` version parsing, too.
Link: https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Link: https://lore.kernel.org/r/20230616001631.463536-11-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
The script already checks if `$RUSTC` and `$BINDGEN` exists via
`command`, but the environment variables may point to a
non-executable file, or the programs may fail for some other reason.
While the script successfully exits with a failure as it should,
the error given can be quite confusing depending on the shell and
the behavior of its `command`. For instance, with `dash`:
$ RUSTC=./mm BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 * + 100 * + "
Thus detect failure exit codes when calling `$RUSTC` and `$BINDGEN` and
print a better message, in a similar way to what we do when extracting
the `libclang` version found by `bindgen`.
Link: https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Link: https://lore.kernel.org/r/20230616001631.463536-10-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
In order to match the version string, `sed` is used in a couple
cases, and `grep` and `head` in a couple others.
Make the script more consistent and easier to understand by
using the same method, `sed`, for all of them.
This makes the version matching also a bit more strict for
the changed cases, since the strings `rustc ` and `bindgen `
will now be required, which should be fine since `rustc`
complains if one attempts to call it with another program
name, and `bindgen` uses a hardcoded string.
In addition, clarify why one of the existing `sed` commands
does not provide an address like the others.
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Link: https://lore.kernel.org/r/20230616001631.463536-9-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
`bindgen`'s output for `libclang`'s version check contains paths, which
in turn may contain strings that look like version numbers [1][2]:
.../6.1.0-dev/.../rust_is_available_bindgen_libclang.h:2:9: warning: clang version 11.1.0 [-W#pragma-messages], err: false
which the script will pick up as the version instead of the latter.
It is also the case that versions may appear after the actual version
(e.g. distribution's version text), which was the reason behind `head` [3]:
.../rust-is-available-bindgen-libclang.h:2:9: warning: clang version 13.0.0 (Fedora 13.0.0-3.fc35) [-W#pragma-messages], err: false
Thus instead ask for a match after the `clang version` string.
Reported-by: Jordan Isaacs <mail@jdisaacs.com>
Closes: https://github.com/Rust-for-Linux/linux/issues/942 [1]
Reported-by: "Ethan D. Twardy" <ethan.twardy@gmail.com>
Closes: https://lore.kernel.org/rust-for-linux/20230528131802.6390-2-ethan.twardy@gmail.com/ [2]
Reported-by: Tiago Lam <tiagolam@gmail.com>
Closes: https://github.com/Rust-for-Linux/linux/pull/789 [3]
Fixes: 78521f3399 ("scripts: add `rust_is_available.sh`")
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Reviewed-by: Ethan Twardy <ethan.twardy@gmail.com>
Tested-by: Ethan Twardy <ethan.twardy@gmail.com>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/r/20230616001631.463536-8-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Sometimes [1] users may attempt to setup the Rust support by
checking what Kbuild does and they end up finding out about
`scripts/rust_is_available.sh`. Inevitably, they run the script
directly, but unless they setup the required variables,
the result of the script is not meaningful.
We could add some defaults to the variables, but that could be
confusing for those that may override the defaults (compared
to their kernel builds), and `$CC` would not be a simple default
in any case.
Therefore, instead, explicitly check whether the expected variables
are set (`$RUSTC`, `$BINDGEN` and `$CC`). If not, print an explanation
about the fact that the script is meant to be called from Kbuild,
since that is the most likely cause for the variables not being set.
Link: https://lore.kernel.org/oe-kbuild-all/Y6r4mXz5NS0+HVXo@zn.tnic/ [1]
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/r/20230616001631.463536-7-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
`scripts/rust_is_available.sh` calls `bindgen` with a special
header in order to check whether the `libclang` version in use
is suitable.
However, the invocation itself may fail if, for instance, `bindgen`
cannot locate `libclang`. This is fine for Kconfig (since the
script will still fail and therefore disable Rust as it should),
but it is pretty confusing for users of the `rustavailable` target
given the error will be unrelated:
./scripts/rust_is_available.sh: 21: arithmetic expression: expecting primary: "100000 * + 100 * + "
make: *** [Makefile:1816: rustavailable] Error 2
Instead, run the `bindgen` invocation independently in a previous
step, saving its output and return code. If it fails, then show
the user a proper error message. Otherwise, continue as usual
with the saved output.
Since the previous patch we show a reference to the docs, and
the docs now explain how `bindgen` looks for `libclang`,
thus the error message can leverage the documentation, avoiding
duplication here (and making users aware of the setup guide in
the documentation).
Reported-by: Nick Desaulniers <ndesaulniers@google.com>
Link: https://lore.kernel.org/rust-for-linux/CAKwvOdm5JT4wbdQQYuW+RT07rCi6whGBM2iUAyg8A1CmLXG6Nw@mail.gmail.com/
Reported-by: François Valenduc <francoisvalenduc@gmail.com>
Closes: https://github.com/Rust-for-Linux/linux/issues/934
Reported-by: Alexandru Radovici <msg4alex@gmail.com>
Closes: https://github.com/Rust-for-Linux/linux/pull/921
Reported-by: Matthew Leach <dev@mattleach.net>
Closes: https://lore.kernel.org/rust-for-linux/20230507084116.1099067-1-dev@mattleach.net/
Fixes: 78521f3399 ("scripts: add `rust_is_available.sh`")
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Reviewed-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/r/20230616001631.463536-6-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
People trying out the Rust support in the kernel may get
warnings and errors from `scripts/rust_is_available.sh`
from the `rustavailable` target or the build step.
Some of those users may be following the Quick Start guide,
but others may not (likely those getting warnings from
the build step instead of the target).
While the messages are fairly clear on what the problem is,
it may not be clear how to solve the particular issue,
especially for those not aware of the documentation.
We could add all sorts of details on the script for each one,
but it is better to point users to the documentation instead,
where it is easily readable in different formats. It also
avoids duplication.
Thus add a reference to the documentation whenever the script
fails or there is at least a warning.
Reviewed-by: Finn Behrens <fin@nyantec.com>
Reviewed-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Link: https://lore.kernel.org/r/20230616001631.463536-5-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
rust_is_available.sh uses cc-version.sh to identify which C compiler is
in use, as scripts/Kconfig.include does. cc-version.sh isn't designed to
be able to handle multiple arguments in one variable, i.e. "ccache clang".
Its invocation in rust_is_available.sh quotes "$CC", which makes
$1 == "ccache clang" instead of the intended $1 == ccache & $2 == clang.
cc-version.sh could also be changed to handle having "ccache clang" as one
argument, but it only has the one consumer upstream, making it simpler to
fix the caller here.
Signed-off-by: Russell Currey <ruscur@russell.cc>
Fixes: 78521f3399 ("scripts: add `rust_is_available.sh`")
Link: https://github.com/Rust-for-Linux/linux/pull/873
[ Reworded title prefix and reflow line to 75 columns. ]
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/r/20230616001631.463536-3-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
The -v option is passed when this script is invoked from Makefile,
but not when invoked from Kconfig.
As you can see in scripts/Kconfig.include, the 'success' macro suppresses
stdout and stderr anyway, so this script does not need to be quiet.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/r/20230109061436.3146442-1-masahiroy@kernel.org
[ Reworded prefix to match the others in the patch series. ]
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Link: https://lore.kernel.org/r/20230616001631.463536-2-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
This script tests whether the Rust toolchain requirements are in place
to enable Rust support. It uses `min-tool-version.sh` to fetch
the version numbers.
The build system will call it to set `CONFIG_RUST_IS_AVAILABLE` in
a later patch.
It also has an option (`-v`) to explain what is missing, which is
useful to set up the development environment. This is used via
the `make rustavailable` target added in a later patch.
Reviewed-by: Kees Cook <keescook@chromium.org>
Co-developed-by: Alex Gaynor <alex.gaynor@gmail.com>
Signed-off-by: Alex Gaynor <alex.gaynor@gmail.com>
Co-developed-by: Wedson Almeida Filho <wedsonaf@google.com>
Signed-off-by: Wedson Almeida Filho <wedsonaf@google.com>
Co-developed-by: Finn Behrens <me@kloenk.de>
Signed-off-by: Finn Behrens <me@kloenk.de>
Co-developed-by: Miguel Cano <macanroj@gmail.com>
Signed-off-by: Miguel Cano <macanroj@gmail.com>
Co-developed-by: Tiago Lam <tiagolam@gmail.com>
Signed-off-by: Tiago Lam <tiagolam@gmail.com>
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
