This fixes https://github.com/llvm/llvm-project/issues/87324.
We haven't been able to come up with a minimal reproducer but we can
reliabely avoid this failure with the following fix. Prior to the
GetGlobalLowLevelAllocator change, the old LowLevelAllocator aquired a
lock associated with it preventing that specific allocator from being
accessed at the same time by many threads. With the
GetGlobalLowLevelAllocator change, I had accidentally replaced it but
not taken into account the lock, so we can have a data race if the
allocator is used at any point while a thread is being created. The
global allocator can be used for flag parsing or registering asan
globals.
This releases the requirement that we need to preserve the memory for
all regions at the beginning. It needs a huge amount of contiguous pages
and which may be a challenge in certain cases. Therefore, adding a new
flag, EnableContiguousRegions, to indicate whether we want to allocate
all the regions next to each other.
Note that once the EnableContiguousRegions is disabled,
EnableRandomOffset becomes irrelevant because the base of each region is
already random.
Unlike the other compiler-rt unit tests MemProf was not using the
`generate_compiler_rt_tests()` helper that ensures the test is compiled
using the test compiler (generally the Clang binary built earlier).
This was exposed by https://github.com/llvm/llvm-project/pull/83088
because it started adding Clang-specific flags to
COMPILER_RT_UNITTEST_CFLAGS if the compiler ID matched "Clang".
This change should fix the buildbots that compile compiler-rt using
a GCC compiler with LLVM_ENABLE_PROJECTS=compiler-rt.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/88074
As discussed here:
https://github.com/llvm/llvm-project/pull/74394#issuecomment-2035264683
An unintentional change of behavior was introduced in #74394
This code introduced in #74394 :
The first time through
* SANITIZER_MIN_OSX_VERSION is not set
* parse -mmacosx-version-min and set MACOSX_VERSION_MIN_FLAG
* Set and cache SANITIZER_MIN_OSX_VERSION
Subsequent times through:
* SANITIZER_MIN_OSX_VERSION is cached
* (BUG!!) you don't parse -mmacosx-version-min, and don't set
MACOSX_VERSION_MIN_FLAG
MACOSX_VERSION_MIN_FLAG is used later in the file on this line:
63c925ca80/compiler-rt/cmake/config-ix.cmake (L517)
Hoisting this assignment outside the if block returns us to the previous
behavior before this commit, while maintaining the flexibility
introduced with the cache variable
Fix since #75481 got reverted.
- Explicitly set BitfieldBits to 0 to avoid uninitialized field member
for the integer checks:
```diff
- llvm::ConstantInt::get(Builder.getInt8Ty(), Check.first)};
+ llvm::ConstantInt::get(Builder.getInt8Ty(), Check.first),
+ llvm::ConstantInt::get(Builder.getInt32Ty(), 0)};
```
- `Value **Previous` was erroneously `Value *Previous` in
`CodeGenFunction::EmitWithOriginalRHSBitfieldAssignment`, fixed now.
- Update following:
```diff
- if (Kind == CK_IntegralCast) {
+ if (Kind == CK_IntegralCast || Kind == CK_LValueToRValue) {
```
CK_LValueToRValue when going from, e.g., char to char, and
CK_IntegralCast otherwise.
- Make sure that `Value *Previous = nullptr;` is initialized (see
1189e87951)
- Add another extensive testcase
`ubsan/TestCases/ImplicitConversion/bitfield-conversion.c`
---------
Co-authored-by: Vitaly Buka <vitalybuka@gmail.com>
Currently, the testsuite uses the default runtimes path to find the
runtimes libraries which may or may not match the just-built runtimes.
This change uses the `-resource-dir` flag for clang whenever
`COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` is set to ensure that we are
actually testing the currently built libraries rather than the ones
bundled with `${COMPILER_RT_TEST_COMPILER}`.
The existing logic works fine when clang and compiler-rt share the same
build directory ``-DLLVM_ENABLE_PROJECTS=clang;compiler-rt`, but when
building compiler-rt separately we need to tell the compiler used for
the tests where it can find the just-built libraries.
This reduces the fixes check-all failures to one in my configuration:
```
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -G Ninja
-DCMAKE_C_COMPILER=$HOME/output/upstream-llvm/bin/clang
-DCMAKE_CXX_COMPILER=$HOME/output/upstream-llvm/bin/clang++
-DCOMPILER_RT_INCLUDE_TESTS=ON
-DLLVM_EXTERNAL_LIT=$HOME/build/upstream-llvm-project-build/bin/llvm-lit
-DLLVM_CMAKE_DIR=$HOME/output/upstream-llvm
-DCOMPILER_RT_DEBUG=OFF
-S $HOME/src/upstream-llvm-project/compiler-rt
-B $HOME/src/upstream-llvm-project/compiler-rt/cmake-build-all-sanitizers
```
Reviewed By: vitalybuka, delcypher, MaskRay
Pull Request: https://github.com/llvm/llvm-project/pull/83088
The HOST_LINK_VERSION is a hardcoded string in Darwin clang that detects
the linker version at configure time. The driver uses this information
to build the correct set of arguments for the linker. This patch detects
the linker version again during compiler-rt configuration and passes it
to the libfuzzer tests. This allows a clang built on a machine with a
new linker to run compiler-rt tests on a machine with an old linker.
rdar://125932376
This patch implements the implicit truncation and implicit sign change
checks for bitfields using UBSan. E.g.,
`-fsanitize=implicit-bitfield-truncation` and
`-fsanitize=implicit-bitfield-sign-change`.
Clean-up all of the calls and remove the redundant == 0 checks.
There is only one small visible change. For non-Android, the memalign
function will now fail if alignment is zero. Before this would have
passed.
`Linux/instrprof-vtable-value-prof.cpp` needs to be built for the test
to run. However, cpp compile & link failed with undefined-ABI error [1].
See original failure in
https://lab.llvm.org/buildbot/#/builders/18/builds/16429
[1]
```
FAIL: Profile-powerpc64 :: Linux/instrprof-vtable-value-prof.cpp (2406 of 2414)
******************** TEST 'Profile-powerpc64 :: Linux/instrprof-vtable-value-prof.cpp' FAILED ********************
Exit Code: 1
Command Output (stderr):
--
RUN: at line 3: /home/buildbots/llvm-external-buildbots/workers/ppc64be-sanitizer/sanitizer-ppc64be/build/build_debug/./bin/clang --driver-mode=g++ -m64 -ldl -fprofile-generate -fuse-ld=lld -O2 -g -fprofile-generate=. -mllvm -enable-vtable-value-profiling /home/buildbots/llvm-external-buildbots/workers/ppc64be-sanitizer/sanitizer-ppc64be/build/llvm-project/compiler-rt/test/profile/Linux/instrprof-vtable-value-prof.cpp -o /home/buildbots/llvm-external-buildbots/workers/ppc64be-sanitizer/sanitizer-ppc64be/build/build_debug/runtimes/runtimes-bins/compiler-rt/test/profile/Profile-powerpc64/Linux/Output/instrprof-vtable-value-prof.cpp.tmp-test
+ /home/buildbots/llvm-external-buildbots/workers/ppc64be-sanitizer/sanitizer-ppc64be/build/build_debug/./bin/clang --driver-mode=g++ -m64 -ldl -fprofile-generate -fuse-ld=lld -O2 -g -fprofile-generate=. -mllvm -enable-vtable-value-profiling /home/buildbots/llvm-external-buildbots/workers/ppc64be-sanitizer/sanitizer-ppc64be/build/llvm-project/compiler-rt/test/profile/Linux/instrprof-vtable-value-prof.cpp -o /home/buildbots/llvm-external-buildbots/workers/ppc64be-sanitizer/sanitizer-ppc64be/build/build_debug/runtimes/runtimes-bins/compiler-rt/test/profile/Profile-powerpc64/Linux/Output/instrprof-vtable-value-prof.cpp.tmp-test
ld.lld: error: /lib/../lib64/Scrt1.o: ABI version 1 is not supported
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
(The profile format change is split into a standalone change into https://github.com/llvm/llvm-project/pull/81691)
* For InstrFDO value profiling, implement instrumentation and lowering for virtual table address.
* This is controlled by `-enable-vtable-value-profiling` and off by default.
* When the option is on, raw profiles will carry serialized `VTableProfData` structs and compressed vtables as payloads.
* Implement profile reader and writer support
* Raw profile reader is used by `llvm-profdata` but not compiler. Raw profile reader will construct InstrProfSymtab with symbol names, and map profiled runtime address to vtable symbols.
* Indexed profile reader is used by `llvm-profdata` and compiler. When initialized, the reader stores a pointer to the beginning of in-memory compressed vtable names and the length of string. When used in `llvm-profdata`, reader decompress the string to show symbols of a profiled site. When used in compiler, string decompression doesn't
happen since IR is used to construct InstrProfSymtab.
* Indexed profile writer collects the list of vtable names, and stores that to index profiles.
* Text profile reader and writer support are added but mostly follow the implementation for indirect-call value type.
* `llvm-profdata show -show-vtables <args> <profile>` is implemented.
rfc in
https://discourse.llvm.org/t/rfc-dynamic-type-profiling-and-optimizations-in-llvm/74600#pick-instrumentation-points-and-instrument-runtime-types-7
This fixes a nit I had accidentally introduced in
https://github.com/llvm/llvm-project/pull/85142
I don't think the value of __msan_get_track_origins() will change
between the start and end of InitShadowWithReExec, but it's cleaner to
use the parameter.
It appears that qemu does not actually cause mmap to fail when calling
setrlimit to limit the address space size. In the two tests that use
setrlimit, detect if mmap still works and skip the tests in that case.
Since all Android targets should support setrlimit, compile out the mmap
check code for them.
Only attempt to initialize the ring buffer when tracking is enabled.
Updated unit tests, and added a few new unit tests to verify the
RingBuffer is not initialized by default.
Verified that the two maps associated with the RingBuffer are not
created in processes by default.
In the StackDepot::isValid function, there is work to validate the
TabMask variable. Unfortunately, if TabMask is set to the maximum
allowed value, TabSize = TabMask + 1 becomes zero and validation passes.
Disallow that case to prevent invalid reads into the Tab structure.
I introduced this test in #86537, let's remove some unneeded debugging
comments.
This PR was initially also moving the test to linux directory but looks
like it's already done by 17ab9e6446 .
This reverts commit 36ca9a2902.
We were using the `time` as the seed while choosing a new TSD. To make
the access of TSDs evenly distributed, we require a higher precision in
`time`. Otherwise, many threads may result in having the same random
access pattern on TSDs because they share the same `time` in certain
period. On Linux, CLOCK_MONOTONIC_COARSE usually adopts 4 ms precision.
This is way higher than the average accessing time of TSD (which is
usually less than 1 us). As a result, when multiple threads try to
select a new TSD in a 4 ms interval, they share the same `time` seed and
end up choosing and congesting on the same TSD.
compiler-rt may depend on libc (memset etc). Likewise a libc built by
clang may depend on compiler-rt builtins.
This circular dependency doesn't matter much once they're both compiled.
The easy compilation order to build both from source is:
1. install libc headers somewhere
2. build compiler-rt builtins against those headers
3. build libc against compiler-rt builtins
This patch relaxes the cmake sanity check to pass without requiring a
libc library. That allows the above sequence to work. Otherwise one
needs to build a static libc, then use that to pass the compiler-rt
cmake check, then build a normal libc.
Do not abort if a vector cannot increase its own capacity. In that case,
push_back calls silently fail.
Modify the ScopedString implementation so that it no longer requires two
passes to do the format. Move the helper functions to be private member
functions so that they can use push_back directly. This allows the
capacity to be increased under the hood and/or silently discards data if
the capacity is exceeded and cannot be increased.
Add new tests for the Vector and ScopedString for capacity increase
failures.
Doing this so that if a map call fails, and we are attempting to write
an error string, we can still get some of the message dumped. This also
avoids crashing in Scudo code, and makes the caller handle any failures.
In glibc versions before 2.33. `libc_nonshared.a` defines
`__fxstat/__fxstat64` but there is no `fstat/fstat64`. glibc 2.33 added
`fstat/fstat64` and obsoleted `__fxstat/__fxstat64`. Ports added after
2.33 do not provide `__fxstat/__fxstat64`, so our `fstat/fstat64`
interceptors using `__fxstat/__fxstat64` interceptors would lead to
runtime failures on such ports (LoongArch and certain RISC-V ports).
Similar to https://reviews.llvm.org/D118423, refine the conditions that
we define fstat{,64} interceptors. `fstat` is supported by musl/*BSD
while `fstat64` is glibc only.
Fixes#83844.
This PR adds callbacks to mark futex syscalls as blocking. Unfortunately
we didn't have a mechanism before to mark syscalls as a blocking call,
so I had to implement it, but it mostly reuses the `BlockingCall`
implementation
[here](96819daa3d/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp (L362-L380)).
The issue includes some information but this issue was discovered
because Rust uses futexes directly. So most likely we need to update
Rust as well to use these callbacks.
Also see the latest comments in #85188 for some context.
I also sent another PR #84162 to mark `pthread_*_lock` calls as
blocking.
Fixes#83561.
When a thread is blocked on a mutex and we send an async signal to that
mutex, it never arrives because tsan thinks that `pthread_mutex_lock` is
not a blocking function. This patch marks `pthread_*_lock` functions as
blocking so we can successfully deliver async signals like `SIGPROF`
when the thread is blocked on them.
See the issue also for more details. I also added a test, which is a
simplified version of the compiler explorer example I posted in the
issue.
Please let me know if you have any other ideas or things to improve!
Happy to work on them.
Also I filed #83844 which is more tricky because we don't have a libc
wrapper for `SYS_futex`. I'm not sure how to intercept this yet. Please
let me know if you have ideas on that as well. Thanks!
/Zl is the equivalent of -nodefaultlibs. The idea here is to make sure
that the asan runtime doesn't have any defaultlibs directives, which
makes it easier to mix an asan runtime built with the dynamic CRT with
an application built with the static CRT (or vise-versa).
This is part of the overall effort to remove the static asan runtime on
windows entirely: https://github.com/llvm/llvm-project/pull/81677
Co-authored-by: Amy Wishnousky <amyw@microsoft.com>
The HOST_LINK_VERSION is a hardcoded string in Darwin clang that detects
the linker version at configure time. The driver uses this information
to build the correct set of arguments for the linker. This patch detects
the linker version again during compiler-rt configuration and passes it
to the tests. This allows a clang built on a machine with a new linker
to run compiler-rt tests on a machine with an old linker.
rdar://125198603
