We didn't save that much boilerplate with those anchors, but it made the
pipeline description harder to understand because the definition of the
jobs and the agents they run on were so far apart. Anchors are useful
for the other common boilerplate we truly share between all jobs, but it
seems reasonable to define agent queues in-line.
It's not that I have much love for C++03, but we should ensure that it
works. Some recent changes broke this configuration because slightly
older Clang versions don't support attribute syntax in C++03 mode.
There are a few drive-by fixes:
- Since the combination RTTI disabled and exceptions enabled do not
work, this combination is prohibited.
- A small NFC in any fixing clang-tidy.
The code in the Buildkite configuration is prepared for using the std
module. There are more fixes needed for that configuration which will be
done in a separate commit.
This removes the Google hosted Linux buildkite builders. We have since
moved all of them over to github actions.
Follow up changes will be sent for android.
Picolibc is a C Standard Library that is commonly used in embedded
environments. This patch adds initial support for this configuration
along with pre-commit CI. As of this patch, the test suite only builds
the tests and nothing is run. A follow-up patch will make the test suite
actually run the tests.
Differential Revision: https://reviews.llvm.org/D154246
The ignore_format.txt file and the associated checks have been causing a
lot of confusion since we introduced them. Formatting becomes one of the
main hurdle for contributors (especially new contributors), and that is
not great.
The original goal of ignore_format.txt was to enforce clang-format only
in a subset of the files of the project. In practice, we have now
shifted to a model where we have a Github action that checks whether new
code surrounding edits is formatted. In that context, it probably
doesn't make sense to keep having a ignore list for formatting files.
After this patch, the clang-format job will enforce that all new code is
formatted properly, and that all edits to existing files are formatted
properly, regardless of which files the edits are in. This seems
reasonable and I believe will lead to much less confusion than our
current setup.
In the future, we could consider clang-formatting the whole code base
once and for all but this requires a bit of upfront technical work to
put in place a merge driver to help resolve merge conflicts across
formatting changes.
Trailing whitespace is removed by clang-format, so if someone tries to
check-in new code with trailing whitespaces, it'll be caught by the
clang-format job. Removing this duplication helps reduce the confusion
around our numerous ways of enforcing formatting rules.
This change adds the image used by the self-hosted Github Actions
builders.
In an attempt to make the transition simple, all of the different images
share as much of the same state as possible, including packages, users,
etc... This results in bigger images, but that shouldn't be a problem.
That said, the refactorings caused the buildkite image to shrink by 100
MB.
This change also renames all of the packages for consistency. Bots will
have to be changed to use the new package names eventually.
Again, docker-compose was used as the source of truth for defining
argument.
I have already pushed example images to ghcr.io/libcxx/<name>:testing
There are three changes present in this PR.
1. Use github packages for libcxx-builder rather than dockerhub.
The ldionne/libcxx-builder image will now be hosted at
ghcr.io/libcxx/libcxx-builder. This has the benefit of allowing members
of the github org to push new versions.
In the future I hope to add github actions to rebuild the image as
needed.
2. Add docker-compose file
The compose file allows to to specify the package repository, so that
users can simply write 'docker compose build' and 'docker compose push'.
It also gives us a centralized place to manage version arguments which
change frequently.
3. Use non-shell CMD form.
This may help the google libcxx builders disconnect more gracefully as
the shell form of CMD may eat the shutdown signal. I'm hoping this
corrects inaccurate agent counts from the buildkite API, since when the
VM's terminate, they do so without signaling it to buildkite, which
hangs around waiting for them to reconnect. It's likely more changes
will be needed though.
---------
Co-authored-by: Mark de Wever <zar-rpg@xs4all.nl>
1. Instead of using individual "boolean" macros, have an "enum" macro
`_LIBCPP_HARDENING_MODE`. This avoids issues with macros being
mutually exclusive and makes overriding the hardening mode within a TU
more straightforward.
2. Rename the safe mode to debug-lite.
This brings the code in line with the RFC:
https://discourse.llvm.org/t/rfc-hardening-in-libc/73925Fixes#65101
This allows testing the upcoming CMake 3.28 release in the CI. CMake
3.28 will have non-experimental support for C++20 modules. So this would
be a better CMake version for the modular builds.
The goal is to remove CMake 3.27 from the CI when the builder work
properly with 3.28.
Add a new test mode that enables the hardened mode in combination with
ABI-breaking changes (only bounded iterators currently) and reenable the
bounded iterator tests for `span` and `string_view`.
There is currently a major problem with the CI output:
The information you need to see never appears in the visible log.
This is because our logs are very verbose, and list (A) every test as
they run it, and (B) every unsupported tests. This can be thousands of
lines.
(A) was introduced by me when I disabled the PTY progress bar, which
doesn't play nice with log files. That change was an improvement, but I
have now disabled the PTY on the builders, so we can go back to passing
`-s`. When `-s` is passed but no PTY is available, it prints a long
friendly progress indicator.
(B) is solved here by disabling the printing of unsupported tests at the
end of the test suite. While it can be useful on occasion to audit the
list of unsupported tests, it's far from a common operation. Instead
people want to see the log of their failure. We should upload the xml
results, so if auditing is needed, it can be done using that.
Hopefully this change will make it so that the test failures appear in
the actual log output
Add a Dockerfile for a new Docker image, libcxx-builder-android, that
extends libcxx-builder with support for testing Android.
The image includes these things:
* An Android Clang compiler and sysroot.
* The Android platform-tools (e.g. adb), so that an Android buildbot can
run programs on an Android device. At container startup, copy these
platform tools to an "android-platform-tools" Docker volume to share
them with an emulator container. This copying ensures that the emulator
and libcxx-builder containers avoid mismatched adb versions.
* Docker, so that an Android buildbot can manage a sibling Docker
container that runs the Android emulator.
Add an Android-specific run-buildbot-container script for local
development. Currently using this script requires building
libcxx-build-android and an emulator image locally.
Fixes: https://github.com/llvm/llvm-project/issues/69270
Differential Revision: https://reviews.llvm.org/D155271
I could probably break this commit into more pieces.
---
This patch adds libc++ support for Android L (Android 5.0+) and up,
tested using the Android team's current compiler, a recent version of
the AOSP sysroot, and the x86[-64] Android Emulator.
CMake and Lit Configuration:
Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake
to cross-compile to Android without using CMake's built-in NDK support
(which only works with an actual packaged NDK).
Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++
(and libc++abi) for Android. This file configures libc++ to match what
the NDK distributes, e.g.:
- libc++_shared.so (includes libc++abi objects, there is no
libc++abi.so). libunwind is linked statically but not exported.
- libc++_static.a (does not include libc++abi) and libc++abi.a
- `std::__ndk1` namespace
- All the libraries are built with `__ANDROID_API__=21`, even when they
are linked to something targeting a higher API level.
(However, when the Android LLVM team builds these components, they do
not use these CMake cache files. Instead they use Python scripts to
configure the builds. See
https://android.googlesource.com/toolchain/llvm_android/.)
Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android
NDK's libc++_shared.so. These files can target old or new Android
devices. The Android LLVM team uses these test files to test libc++ for
both arm/arm64 and x86/x86_64 architectures.
The Android testing mode works by setting %{executor} to adb_run.py,
which uses `adb push` and `adb shell` to run tests remotely. adb_run.py
always runs tests as the "shell" user even on an old emulator where "adb
unroot" doesn't work. The script has workarounds for old Android
devices. The script uses a Unix domain socket on the host
(--job-limit-socket) to restrict concurrent adb invocations. Compiling
the tests is a major part of libc++ testing run-time, so it's desirable
to exploit all the host cores without overburdening the test devices,
which can have far fewer cores.
BuildKite CI:
Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang
and an Android sysroot to build libc++, then starts an Android emulator
container to run tests.
Run the emulator and an adb server in a separate Docker container
(libcxx-ci-android-emulator), and create a separate Docker image for
each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the
container's adb server. Running the only adb server inside the container
makes cleanup more reliable between test runs, e.g. the adb client
doesn't create a `~/.android` directory and the adb server can be
restarted along with the emulator using docker stop/run. (N.B. The
emulator insists on connecting to an adb server and will start one
itself if it can't connect to one.)
The suffix to the android-ndk-* job is a label that concisely specifies
an Android SDK emulator image. e.g.:
- "system-images;android-21;default;x86" ==> 21-def-x86
- "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64
Fixes: https://github.com/llvm/llvm-project/issues/69270
Differential Revision: https://reviews.llvm.org/D139147
This allows running these quick checks faster than in our Buildkite
pipeline, which has much more latency. This will also avoid blocking the
rest of the testing pipeline in case the generated-files checks are
failing.
This updates the clang-format we use in libc++ to 17. This is necessary
to start running the generated-files checks in GitHub Actions (in
#68920). In fact this is a pre-existing issue regardless of #68920 --
right now our ignore_format.txt job disagrees with the LLVM-wide
clang-format job.
The step that checked for ignore_format.txt being consistent with the
tree wouldn't print any explicit diagnostic when failing, which led to
confusion. After this patch, an explicit diagnostic will be printed by
the job along with the required diff to ignore_format.txt.
Since LLVM 17 has been branched and is on the verge of being released,
we can drop the CI job that tests against Clang 15. I think the number
of cherry-picks to `release/17.x` will be a lot smaller now, so keeping
a Clang 15 job around for that purpose seems unnecessary.
As a fly-by, this patch also removes some Clang 15 workarounds and test
suite annotations as we usually do. It also removes some slightly older
gcc test suite annotations that were missed.
Now that we have a LLVM-wide GitHub action, it doesn't make sense to
have a custom job for libc++ in our BuildKite CI pipeline. This only creates
confusion as to which CI job should be acted upon.
We were running the test suite in C++20 mode, but we should really be
tracking the latest supported version. We can't do C++26 because the
latest stable AppleClang doesn't support it yet.
The safe mode is in-between the hardened and the debug modes, extending
the checks contained in the hardened mode with certain checks that are
relatively cheap and prevent common sources of errors but aren't
security-critical. Thus, the safe mode trades off some performance for
a wider set of checks, but unlike the debug mode, it can still be used
in production.
Differential Revision: https://reviews.llvm.org/D158823
This is the first step to implement time zone support in libc++. This
adds the complete tzdb_list class and a minimal tzdb class. The tzdb
class only contains the version, which is used by reload_tzdb.
Next to these classes it contains documentation and build system support
needed for time zone support. The code depends on the IANA Time Zone
Database, which should be available on the platform used or provided by
the libc++ vendors.
The code is labeled as experimental since there will be ABI breaks
during development; the tzdb class needs to have the standard headers.
Implements parts of:
- P0355 Extending <chrono> to Calendars and Time Zones
Addresses:
- LWG3319 Properly reference specification of IANA time zone database
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D154282
Use the same arguments as other builds. This gives better output to
validate what the CI did.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D158860
The vendors of the MSVC STL, libstdc++ and libc++ have agreed [1] to
make the C++23 modules std and std.compat available in C++20. This
provides the std module; libc++ has not implemented the std.compat
module yet.
[1] https://github.com/microsoft/STL/issues/3945
Depends on D158357
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D158358
The summary flag causes a progress bar to appear and be rewritten using
a curses like interface. This works great for interactive sessions, but
produces line after line of garbage in the logs.
This should cause the logs to be much smaller and more readable.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D159120
The std module can be tested in the generic C++23 build. This removes the
special module build.
Note it is possible to enable modules automatically in CMake, but that
would fail in the "parts disabled" builds; they have not been properly
been converted to modules yet.
Depends on D157625
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D157744
This fixes some missing #ifndef and implements the header restrictions
in the modules script.
Depends on D158192
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D158330
