When testing a `clang` upgrade with Rust Binder, Alice encountered [1] a
build failure caused by `bindgen` not translating some symbols related to
tracepoints. This was caused by commit 2e770edd8ce1 ("[libclang] Compute
the right spelling location") changing the behavior of a function exposed
by `libclang`. `bindgen` fixed the regression in commit 600f63895f73
("Use clang_getFileLocation instead of clang_getSpellingLocation").
However, the regression fix is only available in `bindgen` versions
0.69.5 or later (it was backported for 0.69.x). This means that when
older bindgen versions are used with new versions of `libclang`, `bindgen`
may do the wrong thing, which could lead to a build failure.
Alice encountered the bug with some header files related to tracepoints,
but it could also cause build failures in other circumstances. Thus,
always emit a warning when using an old `bindgen` with a new `libclang`
so that other people do not have to spend time chasing down the same
bug.
However, testing just the version is inconvenient, since distributions
do patch their packages without changing the version, so I reduced the
issue into the following piece of code that can trigger the issue:
#define F(x) int x##x
F(foo);
In particular, an unpatched `bindgen` will ignore the macro expansion
and thus not provide a declaration for the exported `int`.
Thus add a build test to `rust_is_available.sh` using the code above
(that is only triggered if the versions appear to be affected), following
what we did for the 0.66.x issue.
Moreover, I checked the status in the major distributions we have
instructions for:
- Fedora 41 was affected but is now OK, since it now ships `bindgen`
0.69.5.
Thanks Ben for the quick reply on the updates that were ongoing.
Fedora 40 and earlier are OK (older `libclang`, and they also now
carry `bindgen` 0.69.5).
- Debian Sid was affected but is now OK, since they now ship a patched
`bindgen` binary (0.66.1-7+b3). The issue was reported to Debian by
email and then as a bug report [2].
Thanks NoisyCoil and Matthias for the quick replies. NoisyCoil handled
the needed updates. Debian may upgrade to `bindgen` 0.70.x, too.
Debian Testing is OK (older `libclang` so far).
- Ubuntu non-LTS (oracular) is affected. The issue was reported to Ubuntu
by email and then as a bug report [3].
Ubuntu LTS is not affected (older `libclang` so far).
- Arch Linux, Gentoo Linux and openSUSE should be OK (newer `bindgen` is
provided). Nix as well (older `libclang` so far).
This issue was also added to our "live list" that tracks issues around
distributions [4].
Cc: Ben Beasley <code@musicinmybrain.net>
Cc: NoisyCoil <noisycoil@tutanota.com>
Cc: Matthias Geiger <werdahias@riseup.net>
Link: https://lore.kernel.org/rust-for-linux/20241030-bindgen-libclang-warn-v1-1-3a7ba9fedcfe@google.com/ [1]
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1086510 [2]
Link: https://bugs.launchpad.net/ubuntu/+source/rust-bindgen-cli/+bug/2086639 [3]
Link: https://github.com/Rust-for-Linux/linux/issues/1127 [4]
Co-developed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Link: https://lore.kernel.org/r/20241111201607.653149-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
`bindgen` versions 0.66.0 and 0.66.1 panic due to C string literals with
NUL characters [1]:
panicked at .cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.66.0/codegen/mod.rs:717:71:
called `Result::unwrap()` on an `Err` value: FromBytesWithNulError { kind: InteriorNul(4) }
Thus, in preparation for supporting several `bindgen` versions, add a
version check to warn the user about it.
Since some distributions may have patched it (e.g. Debian did [2]),
check if that seems to be the case (after the version check matches),
in order to avoid printing a warning in that case.
We could make it an error, but 1) it is going to fail anyway later
in the build, 2) we would disable `RUST`, which is also painful, 3)
someone could have patched it in a way that still makes our extra check
fail (however unlikely), 4) the interior NUL may go away in the headers
(however unlikely). Thus just warn about it so that users know why it
is failing.
In addition, add a couple tests for the new cases.
Link: https://github.com/rust-lang/rust-bindgen/pull/2567 [1]
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069047 [2]
Link: https://lore.kernel.org/r/20240709160615.998336-11-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
With both the workaround for `bindgen` 0.69.0 and the warning about
0.66.0 and 0.66.1 in place, start supporting several `bindgen` versions,
like it was done for the Rust compiler in a previous patch.
All other versions, including the latest 0.69.4, build without errors.
The `bindgen` project, like Rust, has also agreed to have the kernel
in their CI [1] -- thanks! This should help both projects: `bindgen`
will be able to detect early issues like those mentioned above, and the
kernel will be very likely build with new releases (at least for the
basic configuration being tested).
Link: https://github.com/rust-lang/rust-bindgen/pull/2851 [1]
Tested-by: Benno Lossin <benno.lossin@proton.me>
Tested-by: Andreas Hindborg <a.hindborg@samsung.com>
Link: https://lore.kernel.org/r/20240709160615.998336-10-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
`bindgen`'s logic to find `libclang` (via `clang-sys`) may change over
time, and depends on how it was built (e.g. Linux distributions may decide
to build it differently, and we are going to provide documentation on
installing it via distributions later in this series).
Therefore, clarify that `bindgen` may be built in several ways and
simplify the documentation by only mentioning the most prominent
environment variable (`LIBCLANG_PATH`) as an example on how to tweak the
search of the library at runtime (i.e. when `bindgen` is built as our
documentation explains). This also avoids duplicating the documentation,
like `bindgen` itself does (i.e. it refers to `clang-sys`).
Similarly, replace the test we had for this (which used the real program)
with a mocked one, to avoid depending on the particular build as well.
Tested-by: Benno Lossin <benno.lossin@proton.me>
Tested-by: Andreas Hindborg <a.hindborg@samsung.com>
Link: https://lore.kernel.org/r/20240709160615.998336-8-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
It is time to start supporting several Rust compiler versions and thus
establish a minimum Rust version.
We may still want to upgrade the minimum sometimes in the beginning since
there may be important features coming into the language that improve
how we write code (e.g. field projections), which may or may not make
sense to support conditionally.
We will start with a window of two stable releases, and widen it over
time. Thus this patch does not move the current minimum (1.78.0), but
instead adds support for the recently released 1.79.0.
This should already be enough for kernel developers in distributions that
provide recent Rust compiler versions routinely, such as Arch Linux,
Debian Unstable (outside the freeze period), Fedora Linux, Gentoo
Linux (especially the testing channel), Nix (unstable) and openSUSE
Tumbleweed. See the documentation patch about it later in this series.
In addition, Rust for Linux is now being built-tested in Rust's pre-merge
CI [1]. That is, every change that is attempting to land into the Rust
compiler is tested against the kernel, and it is merged only if it passes
-- thanks to the Rust project for that!
Thus, with the pre-merge CI in place, both projects hope to avoid
unintentional changes to Rust that break the kernel. This means that,
in general, apart from intentional changes on their side (that we will
need to workaround conditionally on our side), the upcoming Rust compiler
versions should generally work.
For instance, currently, the beta (1.80.0) and nightly (1.81.0) branches
work as well.
Of course, the Rust for Linux CI job in the Rust toolchain may still need
to be temporarily disabled for different reasons, but the intention is
to help bring Rust for Linux into stable Rust.
Link: https://github.com/rust-lang/rust/pull/125209 [1]
Reviewed-by: Finn Behrens <me@kloenk.dev>
Tested-by: Benno Lossin <benno.lossin@proton.me>
Tested-by: Andreas Hindborg <a.hindborg@samsung.com>
Link: https://lore.kernel.org/r/20240709160615.998336-7-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
The `rust_is_available.sh` script runs for everybody compiling the
kernel, even if not using Rust. Therefore, it is important to ensure
that the script is correct to avoid breaking people's compilation.
In addition, the script needs to be able to handle a set of subtle
cases, including parsing version strings of different tools.
Therefore, maintenance of this script can be greatly eased with
a set of tests.
Thus add a test suite to cover hopefully most of the setups that
the script may encounter in the wild. Extra setups can be easily
added later on if missing.
The script currently covers all the branches of the shell script,
including several ways in which they may be entered.
Python is used for this script, since the script under test
does not depend on Rust, thus hopefully making it easier for others
to use if the need arises.
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Link: https://lore.kernel.org/r/20230616001631.463536-12-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
