In order to test the LLDB data formatters make is required and SWIG
needs to be updated to version 4.
As drive-by, this patch sorts the entries and removes some duplicates.
The OSS fuzz build has been broken for a while because of changes
to the cmake configuration. This patch fixes the issues by defaulting to
libunwind as the default runtime
The documentation mentions manually pushing Docker images to the CI. The
preferred way is to use the proper GitHub action. This updates the
documentation.
---------
Co-authored-by: Will Hawkins <whh8b@obs.cr>
Yesterday, one of the issues the libc++ builders encountered was that
they were using a client that was too old; too old to even update
automatically.
To get things working, i had to push a testing image with this change.
The testing image has been working for 12 hours now, so it's time to
commit to it :-)
Revert "Revert #76246 and #76083"
This reverts commit 5c150e7eeb.
Adds a small fix that should properly disable the tests on Windows.
Unfortunately the original poster has not provided feedback and the
original patch did not fail in the LLVM CI infrastructure.
Modules are known to fail on Windows due to non compliance of the
C library. Currently not having this patch prevents testing on other
platforms.
This removes the entire modules testing infrastructure.
The current infrastructure uses CMake to generate the std and std.compat
module. This requires quite a bit of plumbing and uses CMake. Since
CMake introduced module support in CMake 3.26, modules have a higher
CMake requirement than the rest of the LLVM project. (The LLVM project
requires 3.20.) The main motivation for this approach was how libc++
generated its modules. Every header had its own module partition. This
was changed to improve performance and now only two modules remain. The
code to build these can be manually crafted.
A followup patch will reenable testing modules, using a different
approach.
The updated picolibc version has "isblank" function with external
linkage. This is required for C++ modules support.
This should solve all the problems reported in #76980, but
we'll wait to validate this with the modules build without
closing that issue.
I recently came across LIBCXXABI_USE_LLVM_UNWINDER and was surprised to
notice it was disabled by default. Since we build libunwind by default
and ship it in the LLVM toolchain, it would seem to make sense that
libc++ and libc++abi rely on libunwind for unwinding instead of using
the system-provided unwinding library (if any).
Most importantly, using the system unwinder implies that libc++abi is
ABI compatible with that system unwinder, which is not necessarily the
case. Hence, it makes a lot more sense to instead default to using the
known-to-be-compatible LLVM unwinder, and let vendors manually select a
different unwinder if desired.
As a follow-up change, we should probably apply the same default to
compiler-rt.
Differential Revision: https://reviews.llvm.org/D150897Fixes#77662
rdar://120801778
This patch adds a configuration of the libc++ test suite that enables
optimizations when building the tests. It also adds a new CI
configuration to exercise this on a regular basis. This is added in the
context of [1], which requires building with optimizations in order to
hit the bug.
[1]: https://github.com/llvm/llvm-project/issues/68552
We didn't save that much boilerplate with those anchors, but it made the
pipeline description harder to understand because the definition of the
jobs and the agents they run on were so far apart. Anchors are useful
for the other common boilerplate we truly share between all jobs, but it
seems reasonable to define agent queues in-line.
It's not that I have much love for C++03, but we should ensure that it
works. Some recent changes broke this configuration because slightly
older Clang versions don't support attribute syntax in C++03 mode.
There are a few drive-by fixes:
- Since the combination RTTI disabled and exceptions enabled do not
work, this combination is prohibited.
- A small NFC in any fixing clang-tidy.
The code in the Buildkite configuration is prepared for using the std
module. There are more fixes needed for that configuration which will be
done in a separate commit.
This removes the Google hosted Linux buildkite builders. We have since
moved all of them over to github actions.
Follow up changes will be sent for android.
Picolibc is a C Standard Library that is commonly used in embedded
environments. This patch adds initial support for this configuration
along with pre-commit CI. As of this patch, the test suite only builds
the tests and nothing is run. A follow-up patch will make the test suite
actually run the tests.
Differential Revision: https://reviews.llvm.org/D154246
The ignore_format.txt file and the associated checks have been causing a
lot of confusion since we introduced them. Formatting becomes one of the
main hurdle for contributors (especially new contributors), and that is
not great.
The original goal of ignore_format.txt was to enforce clang-format only
in a subset of the files of the project. In practice, we have now
shifted to a model where we have a Github action that checks whether new
code surrounding edits is formatted. In that context, it probably
doesn't make sense to keep having a ignore list for formatting files.
After this patch, the clang-format job will enforce that all new code is
formatted properly, and that all edits to existing files are formatted
properly, regardless of which files the edits are in. This seems
reasonable and I believe will lead to much less confusion than our
current setup.
In the future, we could consider clang-formatting the whole code base
once and for all but this requires a bit of upfront technical work to
put in place a merge driver to help resolve merge conflicts across
formatting changes.
Trailing whitespace is removed by clang-format, so if someone tries to
check-in new code with trailing whitespaces, it'll be caught by the
clang-format job. Removing this duplication helps reduce the confusion
around our numerous ways of enforcing formatting rules.
This change adds the image used by the self-hosted Github Actions
builders.
In an attempt to make the transition simple, all of the different images
share as much of the same state as possible, including packages, users,
etc... This results in bigger images, but that shouldn't be a problem.
That said, the refactorings caused the buildkite image to shrink by 100
MB.
This change also renames all of the packages for consistency. Bots will
have to be changed to use the new package names eventually.
Again, docker-compose was used as the source of truth for defining
argument.
I have already pushed example images to ghcr.io/libcxx/<name>:testing
There are three changes present in this PR.
1. Use github packages for libcxx-builder rather than dockerhub.
The ldionne/libcxx-builder image will now be hosted at
ghcr.io/libcxx/libcxx-builder. This has the benefit of allowing members
of the github org to push new versions.
In the future I hope to add github actions to rebuild the image as
needed.
2. Add docker-compose file
The compose file allows to to specify the package repository, so that
users can simply write 'docker compose build' and 'docker compose push'.
It also gives us a centralized place to manage version arguments which
change frequently.
3. Use non-shell CMD form.
This may help the google libcxx builders disconnect more gracefully as
the shell form of CMD may eat the shutdown signal. I'm hoping this
corrects inaccurate agent counts from the buildkite API, since when the
VM's terminate, they do so without signaling it to buildkite, which
hangs around waiting for them to reconnect. It's likely more changes
will be needed though.
---------
Co-authored-by: Mark de Wever <zar-rpg@xs4all.nl>
1. Instead of using individual "boolean" macros, have an "enum" macro
`_LIBCPP_HARDENING_MODE`. This avoids issues with macros being
mutually exclusive and makes overriding the hardening mode within a TU
more straightforward.
2. Rename the safe mode to debug-lite.
This brings the code in line with the RFC:
https://discourse.llvm.org/t/rfc-hardening-in-libc/73925Fixes#65101
This allows testing the upcoming CMake 3.28 release in the CI. CMake
3.28 will have non-experimental support for C++20 modules. So this would
be a better CMake version for the modular builds.
The goal is to remove CMake 3.27 from the CI when the builder work
properly with 3.28.
Add a new test mode that enables the hardened mode in combination with
ABI-breaking changes (only bounded iterators currently) and reenable the
bounded iterator tests for `span` and `string_view`.
There is currently a major problem with the CI output:
The information you need to see never appears in the visible log.
This is because our logs are very verbose, and list (A) every test as
they run it, and (B) every unsupported tests. This can be thousands of
lines.
(A) was introduced by me when I disabled the PTY progress bar, which
doesn't play nice with log files. That change was an improvement, but I
have now disabled the PTY on the builders, so we can go back to passing
`-s`. When `-s` is passed but no PTY is available, it prints a long
friendly progress indicator.
(B) is solved here by disabling the printing of unsupported tests at the
end of the test suite. While it can be useful on occasion to audit the
list of unsupported tests, it's far from a common operation. Instead
people want to see the log of their failure. We should upload the xml
results, so if auditing is needed, it can be done using that.
Hopefully this change will make it so that the test failures appear in
the actual log output
Add a Dockerfile for a new Docker image, libcxx-builder-android, that
extends libcxx-builder with support for testing Android.
The image includes these things:
* An Android Clang compiler and sysroot.
* The Android platform-tools (e.g. adb), so that an Android buildbot can
run programs on an Android device. At container startup, copy these
platform tools to an "android-platform-tools" Docker volume to share
them with an emulator container. This copying ensures that the emulator
and libcxx-builder containers avoid mismatched adb versions.
* Docker, so that an Android buildbot can manage a sibling Docker
container that runs the Android emulator.
Add an Android-specific run-buildbot-container script for local
development. Currently using this script requires building
libcxx-build-android and an emulator image locally.
Fixes: https://github.com/llvm/llvm-project/issues/69270
Differential Revision: https://reviews.llvm.org/D155271
I could probably break this commit into more pieces.
---
This patch adds libc++ support for Android L (Android 5.0+) and up,
tested using the Android team's current compiler, a recent version of
the AOSP sysroot, and the x86[-64] Android Emulator.
CMake and Lit Configuration:
Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake
to cross-compile to Android without using CMake's built-in NDK support
(which only works with an actual packaged NDK).
Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++
(and libc++abi) for Android. This file configures libc++ to match what
the NDK distributes, e.g.:
- libc++_shared.so (includes libc++abi objects, there is no
libc++abi.so). libunwind is linked statically but not exported.
- libc++_static.a (does not include libc++abi) and libc++abi.a
- `std::__ndk1` namespace
- All the libraries are built with `__ANDROID_API__=21`, even when they
are linked to something targeting a higher API level.
(However, when the Android LLVM team builds these components, they do
not use these CMake cache files. Instead they use Python scripts to
configure the builds. See
https://android.googlesource.com/toolchain/llvm_android/.)
Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android
NDK's libc++_shared.so. These files can target old or new Android
devices. The Android LLVM team uses these test files to test libc++ for
both arm/arm64 and x86/x86_64 architectures.
The Android testing mode works by setting %{executor} to adb_run.py,
which uses `adb push` and `adb shell` to run tests remotely. adb_run.py
always runs tests as the "shell" user even on an old emulator where "adb
unroot" doesn't work. The script has workarounds for old Android
devices. The script uses a Unix domain socket on the host
(--job-limit-socket) to restrict concurrent adb invocations. Compiling
the tests is a major part of libc++ testing run-time, so it's desirable
to exploit all the host cores without overburdening the test devices,
which can have far fewer cores.
BuildKite CI:
Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang
and an Android sysroot to build libc++, then starts an Android emulator
container to run tests.
Run the emulator and an adb server in a separate Docker container
(libcxx-ci-android-emulator), and create a separate Docker image for
each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the
container's adb server. Running the only adb server inside the container
makes cleanup more reliable between test runs, e.g. the adb client
doesn't create a `~/.android` directory and the adb server can be
restarted along with the emulator using docker stop/run. (N.B. The
emulator insists on connecting to an adb server and will start one
itself if it can't connect to one.)
The suffix to the android-ndk-* job is a label that concisely specifies
an Android SDK emulator image. e.g.:
- "system-images;android-21;default;x86" ==> 21-def-x86
- "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64
Fixes: https://github.com/llvm/llvm-project/issues/69270
Differential Revision: https://reviews.llvm.org/D139147
This allows running these quick checks faster than in our Buildkite
pipeline, which has much more latency. This will also avoid blocking the
rest of the testing pipeline in case the generated-files checks are
failing.
This updates the clang-format we use in libc++ to 17. This is necessary
to start running the generated-files checks in GitHub Actions (in
#68920). In fact this is a pre-existing issue regardless of #68920 --
right now our ignore_format.txt job disagrees with the LLVM-wide
clang-format job.
The step that checked for ignore_format.txt being consistent with the
tree wouldn't print any explicit diagnostic when failing, which led to
confusion. After this patch, an explicit diagnostic will be printed by
the job along with the required diff to ignore_format.txt.
Since LLVM 17 has been branched and is on the verge of being released,
we can drop the CI job that tests against Clang 15. I think the number
of cherry-picks to `release/17.x` will be a lot smaller now, so keeping
a Clang 15 job around for that purpose seems unnecessary.
As a fly-by, this patch also removes some Clang 15 workarounds and test
suite annotations as we usually do. It also removes some slightly older
gcc test suite annotations that were missed.
Now that we have a LLVM-wide GitHub action, it doesn't make sense to
have a custom job for libc++ in our BuildKite CI pipeline. This only creates
confusion as to which CI job should be acted upon.
