Currently, the testsuite uses the default runtimes path to find the
runtimes libraries which may or may not match the just-built runtimes.
This change uses the `-resource-dir` flag for clang whenever
`COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` is set to ensure that we are
actually testing the currently built libraries rather than the ones
bundled with `${COMPILER_RT_TEST_COMPILER}`.
The existing logic works fine when clang and compiler-rt share the same
build directory ``-DLLVM_ENABLE_PROJECTS=clang;compiler-rt`, but when
building compiler-rt separately we need to tell the compiler used for
the tests where it can find the just-built libraries.
This reduces the fixes check-all failures to one in my configuration:
```
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -G Ninja
-DCMAKE_C_COMPILER=$HOME/output/upstream-llvm/bin/clang
-DCMAKE_CXX_COMPILER=$HOME/output/upstream-llvm/bin/clang++
-DCOMPILER_RT_INCLUDE_TESTS=ON
-DLLVM_EXTERNAL_LIT=$HOME/build/upstream-llvm-project-build/bin/llvm-lit
-DLLVM_CMAKE_DIR=$HOME/output/upstream-llvm
-DCOMPILER_RT_DEBUG=OFF
-S $HOME/src/upstream-llvm-project/compiler-rt
-B $HOME/src/upstream-llvm-project/compiler-rt/cmake-build-all-sanitizers
```
Reviewed By: vitalybuka, delcypher, MaskRay
Pull Request: https://github.com/llvm/llvm-project/pull/83088
Since this standalone build configuration uses the runtime libraries that
are being built just now, we need to ensure that e.g. the TSan unit tests
depend on the tsan runtime library. Also fix TSAN_DEPS being overridden
to not include the tsan runtime (commit .....).
This change fixes a build race seen in the CI checks for
TsanRtlTest-x86_64-Test in https://github.com/llvm/llvm-project/pull/83088.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83650
With https://github.com/llvm/llvm-project/pull/83088, we now need the
runtimes to be built before running test if
COMPILER_RT_TEST_STANDALONE_BUILD_LIBS is true, since otherwise we
get failures running `ninja check-all` such as the following:
```
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.fuzzer-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-basic-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-fdr-x86_64.a: No such file or directory
```
This is a follow-up to 058e9b03 which started removing these checks
and it should make it easier to stop forcing COMPILER_RT_STANDALONE_BUILD
for runtimes builds in the future.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83651
The watchOS and tvOS sanitizers do not compile with publicly-available
SDKs, failing on calls such as fork() and posix_spawn(). Updating the
darwin_test_archs test program to include a call to fork() allows cmake
to exclude those platforms when compiling runtimes. This allows public
builds to enable watchOS/tvOS and compile builtins but not sanitizers.
I was getting build failures due to missing <cstddef> when building the
libfuzzer tests. It turns out that the custom command was using
COMPILER_RT_TEST_COMPILER when building the source file rather than
the COMPILER_RT_TEST_CXX_COMPILER.
Pull Request: https://github.com/llvm/llvm-project/pull/83090
When the top-level CMake invocation has `CMAKE_VERBOSE_MAKEFILE=ON`,
indicating the user wants to have verbose builds (i.e. all executed
commands explicitly echoed), some of the subprojects and runtimes (such
as compiler-rt, libcxx, etc) do not build in verbose mode. For example,
with Ninja:
```
[ 99% 6252/6308] cd /build/runtimes/builtins-bins && /usr/local/bin/cmake --build .
[ 0% 6/308] Building C object CMakeFiles/clang_rt.builtins-i386.dir/absvti2.c.o
[ 0% 7/308] Building C object CMakeFiles/clang_rt.builtins-i386.dir/absvdi2.c.o
[ 0% 8/308] Building C object CMakeFiles/clang_rt.builtins-i386.dir/absvsi2.c.o
...
```
This is because `llvm_ExternalProject_Add()` and `add_custom_libcxx()`
use CMake's `ExternalProject_Add()` function to configure such
subproject builds, and do not pass through the `CMAKE_VERBOSE_MAKEFILE`
setting.
Similar to what is done in `clang/CMakeLists.txt`, add
`-DCMAKE_VERBOSE_MAKEFILE=ON` to the `ExternalProject_Add()` invocations
in `llvm_ExternalProject_Add()` and `add_custom_libcxx()`, whenever the
top-level CMake invocation had `CMAKE_VERBOSE_MAKEFILE` turned on.
This can be used to configure runtimes builds (instead of setting flags
individually), and we need to pass it down to the custom libc++ build
for it to work correctly.
The Android LLVM build system builds the arm64 fuzzer lib without
HWASan, but then applications that enable HWASan can generated an object
file with a HWASan-ified version of some libc++ symbols (e.g.
`std::__1::piecewise_construct`). The linker can choose the HWASan-ified
definition, but then it cannot resolve the relocation from
libclang_rt.fuzzer-aarch64-android.a to this symbol because the high
bits of the address are unexpectedly set. This produces an error:
```
relocation R_AARCH64_ADR_PREL_PG_HI21 out of range
```
Fix this problem by linking a custom isolated libc++ into Android's
fuzzer library.
We need to pass through ANDROID_NATIVE_API_LEVEL so that the libc++ for
32-bit Android (API < 24) uses LLVM_FORCE_SMALLFILE_FOR_ANDROID.
Implements for sv39 and sv48 VMA layout.
Userspace only has access to the bottom half of vma range. The top half
is used by kernel. There is no dedicated vsyscall or heap segment.
PIE program is allocated to start at TASK_SIZE/3*2. Maximum ASLR is
ARCH_MMAP_RND_BITS_MAX+PAGE_SHIFT=24+12=36 Loader, vdso and other
libraries are allocated below stack from the top.
Also change RestoreAddr to use 4 bits to accommodate MappingRiscv64_48
Reviewed by: MaskRay, dvyukov, asb, StephenFan, luismarques, jrtc27,
hiraditya, vitalybuka
Differential Revision: https://reviews.llvm.org/D145214
D145214 was reverted because one file was missing in the latest commit.
Luckily the file was there in the previous commit, probably the author
missed uploading that file with latest commit.
Co-authored-by: Alex Fan <alex.fan.q@gmail.com>
Resolved issue with green dragon build by fixing relocations for
MachO/Darwin which doesn't compile without @page/@pageoff directives.
Also silenced a warning about constructor(90) priority being < 101,
which is reserved for the implementation. In this case, we're compiling
the implementation so we should be able to use 90.
This reverts commit 072713add4.
When compiling for SME and using the attributes to use PSTATE.ZA,
Clang will emit calls to SME ABI support routines to save and
restore ZA state.
Reviewed By: paulwalker-arm
Differential Revision: https://reviews.llvm.org/D154045
Implements for sv39 and sv48 VMA layout.
Userspace only has access to the bottom half of vma range. The top half is used by kernel.
There is no dedicated vsyscall or heap segment.
PIE program is allocated to start at TASK_SIZE/3*2. Maximum ASLR is ARCH_MMAP_RND_BITS_MAX+PAGE_SHIFT=24+12=36
Loader, vdso and other libraries are allocated below stack from the top.
Also change RestoreAddr to use 4 bits to accommodate MappingRiscv64_48
Reviewed by: MaskRay, dvyukov, asb, StephenFan, luismarques, jrtc27, hiraditya, vitalybuka
Differential Revision: https://reviews.llvm.org/D145214
Since jitlink for ppc64le is ready for general use, test cases in compiler-rt for ELFNixPlatform support can be enabled.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D156399
Make minor changes to enable DFSAN and its tests on
loongarch64. And port Linux loongarch64 memory mappings
from msan.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D140690
This patch adds basic memory sanitizer support for loongarch64
with 47-bit VMA, which memory layout is based on x86_64.
The LLVM part of the LoongArch memory sanitizer implementation will
be done separately, which will fix failing tests in check-msan.
These failing tests fail with the following same error: "error in
backend: unsupported architecture".
Reviewed By: #sanitizers, vitalybuka, MaskRay
Differential Revision: https://reviews.llvm.org/D140528
Instead of dumping all sources into RTXray object library with a weird
special case for x86, handle multiarch builds better. Build a separate
object library for each arch with its arch-specific sources, then link
in all those libraries.
This fixes the build on platforms that produce fat binaries, such as new
macOS which expects both x86_64 and aarch64 objects in the same library
since Apple Silicon is a thing.
This only enables building XRay support for Apple Silicon. It does not
actually work yet on macOS, neither on Intel nor on Apple Silicon CPUs.
Thus the tests are still disabled.
Reviewed By: MaskRay, phosek
Differential Revision: https://reviews.llvm.org/D153221
CMake older than 3.20.0 is no longer supported.
This removes work-arounds for no longer supported versions.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D152102
Moved setting supported architecture to parent cmake configuration files
so they can be read by both lib and test CMakeList.txt.
Fixed issue with check-asan-abi that did not filter for current host architecture
which caused x86_64 bots to run Arm64 tests.
Added x86_64 as a supported arch to the test cmake file.
rdar://110017569
Differential Revision: https://reviews.llvm.org/D151846
I tried to build libFuzzer for RISC-V and succeeded. All the libFuzzer
targets were successfully built. I tested this on the small hello world code
with a few branches to check the instrumentation; all of them were covered by
libFuzzer on RISC-V arch. So I suppose it makes sense to enable libFuzzer
build for RISC-V.
Reviewed By: phosek, thetruestblue, MaskRay
Differential Revision: https://reviews.llvm.org/D147788
Previously, an .exe suffix was only added for MSVC configurations.
In practice, an .exe suffix is added implicitly by MinGW toolchains
if the output is a suffixless file name. However this can cause lots
of subtle build system confusion, when it's not generating the file it
expected.
Differential Revision: https://reviews.llvm.org/D149029
If any LLVM subprojects are built separately, the LLVM build directory
LLVM_LIBRARY_DIR is added to both the build and install runpaths in
llvm_setup_rpath(), which is incorrect when installed. Separate the
build and install runpaths on ELF platforms and finally remove the
incorrect call to this function for compiler-rt, as previously attempted
in 21c008d5a5. That prior attempt was reverted in 959dbd1761, where it
was said to break the build on macOS and Windows, so I made sure to keep
those platforms the same.
Two examples of incorrect runpaths that are currently added, one from
the latest LLVM 16 toolchain for linux x86_64:
> readelf -d clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.*so | ag "File:|runpath"
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.dyndd.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.hwasan_aliases.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.hwasan.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.memprof.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.scudo_standalone.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.tsan.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.ubsan_minimal.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
File: clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/lib/x86_64-unknown-linux-gnu/libclang_rt.ubsan_standalone.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/tmp/llvm_release/final/Phase3/Release/llvmCore-16.0.0-final.obj/./lib]
Another is in the Swift toolchain, which builds lldb separately:
> readelf -d swift-5.9-DEVELOPMENT-SNAPSHOT-2023-03-24-a-ubuntu20.04/usr/{bin/lldb*,lib/liblldb.so}|ag "File:|runpath"
File: swift-5.9-DEVELOPMENT-SNAPSHOT-2023-03-24-a-ubuntu20.04/usr/bin/lldb
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/home/build-user/build/buildbot_linux/llvm-linux-x86_64/./lib]
File: swift-5.9-DEVELOPMENT-SNAPSHOT-2023-03-24-a-ubuntu20.04/usr/bin/lldb-argdumper
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/home/build-user/build/buildbot_linux/llvm-linux-x86_64/./lib]
File: swift-5.9-DEVELOPMENT-SNAPSHOT-2023-03-24-a-ubuntu20.04/usr/bin/lldb-server
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/home/build-user/build/buildbot_linux/llvm-linux-x86_64/./lib]
File: swift-5.9-DEVELOPMENT-SNAPSHOT-2023-03-24-a-ubuntu20.04/usr/lib/liblldb.so
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:/home/build-user/build/buildbot_linux/llvm-linux-x86_64/./lib:/home/build-user/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux:$ORIGIN/../lib/swift/linux]
This patch should fix this problem of absolute paths from the build host
leaking out into the toolchain's runpaths.
Differential revision: https://reviews.llvm.org/D146918
Fixes the following on cmake version 3.24.2:
'tools/clang/runtime/compiler-rt-stamps/compiler-rt-source_dirinfo.txt',
needed by 'tools/clang/runtime/compiler-rt-stamps/compiler-rt-download',
missing and no known rule to make it
Maybe related to https://cmake.org/cmake/help/latest/release/3.24.html#modules
COMPILER_RT_DEFAULT_TARGET_ONLY is the goto workaround for building
only one of a multilib setup while still being able to autodetect the native target.
For example only building 64 bit libraries on Intel instead of 32 and 64 bit.
Which may fail without 32 bit compatibility libs installed.
As reported in https://discourse.llvm.org/t/configuring-compiler-rt-to-use-default-target-only-does-not-work/62727
this build config hasn't worked in a while.
It relies on a variable usually set during cross compliation,
though my testing shows you can set it in other scenarios too.
https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_TARGET.html
So the options you need to provide are:
-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON -DCMAKE_C_COMPILER_TARGET="aarch64-unknown-linux-gnu"
And we should error if CMAKE_C_COMPILER_TARGET isn't set.
Reviewed By: glaubitz
Differential Revision: https://reviews.llvm.org/D147598
-Werror is redundant because CMake not only checks the
exit status, but also parses the output of try_compile.
-Wformat-security is a special case, because specifying it
without -Wformat causes gcc to emit a warning in a format
not recognized by CMake, which makes the check pass.
Enable Scudo on RISCV64 on both clang side and compiler-rt side.
Reviewers: cryptoad, eugenis, vitalybuka, luismarques, hiraditya
Reviewed By: vitalybuka, luismarques
Differential Revision: https://reviews.llvm.org/D137397
clang-cl doesn't support -dumpmachine directly, so we need to
preface it with /clang: in order to get this probing function
to work.
This is needed in order to run cmake directly on the runtimes
directory.
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D143557
The orginal single folder layout produced libraries in the form:
lib/linux/<libname>-<archname>.a
That archname for Arm depended on whether you had hard or soft float.
This is sometimes shown as "arm" (soft) vs. "armhf" (hard).
If this is set in a triple we do it in the final portion, the ABI.
"gnueabi" for soft float, "gnueabihf" for hard float.
Meaning that the expected triple is:
arm-unknown-linux-gnueabihf
Not this:
armhf-unknown-linux-gnueabihf
For the per target layout I have decided to rely on the ABI portion
of the triple instead of the arch name used for the old layout
(doing that would produce the invalid triple above).
This means that building with triple:
armv8l-unknown-linux-gnueabihf
Will result in libraries in:
lib/arm-unknown-linux-gnueabihf/
And clang will now know to look for "arm" instead of "armv8l".
Meaning that we can share libraries between an armv8 and armv7 build
as we did with the previous layout. In addition to handling spelling
differences e.g. "armv8l" with an "l" on some Linux distros.
compiler-rt will autodetect that the "armhf" and/or "arm" architecture
can be built. We then replace the given triple's architecture with that.
Then if the triple's float ABI doesn't match, we change that. That new
triple is then used as the folder name.
If you select to build only the given triple, with COMPILER_RT_DEFAULT_TARGET_ONLY,
compiler-rt will not autodetect the architecture and for that I assume you
know what you're doing. In that case the library path will use the unomdified triple.
From what I can tell, this is how most large builds e.g Android and
Arm's Embedded Toolchain for llvm do things. I assume that big endian "armeb"
builds would end up doing this too.
Bare metal builds will not be using per target runtime dirs so they
remain as they were.
Depends on D139536
Reviewed By: MaskRay, phosek
Differential Revision: https://reviews.llvm.org/D140011
Reviewed by: vitalybuka, kito-cheng
These changes will allow to use HWASAN with RISCV64 architecture.
The majority of existing tests are passing with a few exceptions(see below).
The tests were running on QEMU, since currently there're no hardware with support
for J-extension.
For this feature to work the system must support PR_{SET,GET}_TAGGED_ADDR_CTRL
syscall. For now this is only available for a patched Linux kernel and QEMU with
enabled experimental J-extension.
Results of running HWASAN lit tests for RISC-V:
```
Unsupported : 6
Passed : 79
Expectedly Failed: 2
```
Tests are marked as expected to fail or unsupported either because of:
- android platform not being supported
- no support for legacy hwasan v1 mode
- test config explicitly uses aarch64 option
```
UNSUPPORTED: HWAddressSanitizer-riscv64 :: TestCases/abort-message-android.cpp
UNSUPPORTED: HWAddressSanitizer-riscv64 :: TestCases/cfi.cpp
UNSUPPORTED: HWAddressSanitizer-riscv64 :: TestCases/lto.c
UNSUPPORTED: HWAddressSanitizer-riscv64 :: TestCases/print-memory-usage-android.c
UNSUPPORTED: HWAddressSanitizer-riscv64 :: TestCases/register-dump-read.c
UNSUPPORTED: HWAddressSanitizer-riscv64 :: TestCases/try-catch.cpp
XFAIL: HWAddressSanitizer-riscv64 :: TestCases/stack-oob.c
XFAIL: HWAddressSanitizer-riscv64 :: TestCases/exported-tagged-global.c
```
Differential Revision: https://reviews.llvm.org/D131345
