I introduced this test in #86537, let's remove some unneeded debugging
comments.
This PR was initially also moving the test to linux directory but looks
like it's already done by 17ab9e6446 .
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
v4:
- Add interface attribute to __sanitizer_internal_mem* declarations as
well, as otherwise some compilers (MSVC) will complain.
- Add SANITIZER_COMMON_NO_REDEFINE_BUILTINS to source files using
C++STL, since this could lead to ODR violations (see added comment).
v3:
- Don't use ALIAS() to alias internal_mem*() functions to
__sanitizer_internal_mem*() functions, but just define them as
ALWAYS_INLINE functions instead. This will work on darwin and windows.
v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
v3:
- Don't use ALIAS() to alias internal_mem*() functions to
__sanitizer_internal_mem*() functions, but just define them as
ALWAYS_INLINE functions instead. This will work on darwin and windows.
v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
This is an ongoing series of commits that are reformatting our
Python code. This catches the last of the python files to
reformat. Since they where so few I bunched them together.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: jhenderson, #libc, Mordante, sivachandra
Differential Revision: https://reviews.llvm.org/D150784
This patch adds mmap_56bit_test.c, which attempts to deliberately
mmap above 2^47. This currently exposes a failure on ASan on
systems with 56-bit user address spaces, because it allows an mmap
above 2^47 but the shadow mapping is only set up for a 2^47 user address
space.
Additionally, this patch moves pie_no_aslr from the tsan tests into
sanitizer-common, for greater test coverage.
Differential Revision: https://reviews.llvm.org/D149176
When trying to debug some `compiler-rt` unittests, I initially had a hard
time because
- even in a `Debug` build one needs to set `COMPILER_RT_DEBUG` to get
debugging info for some of the code and
- even so the unittests used a hardcoded `-O2` which often makes debugging
impossible.
This patch addresses this by instead using `-O0` if `COMPILER_RT_DEBUG`.
Changes relative to the previous commit:
- Use `string(APPEND)` for `COMPILER_RT_TEST_COMPILER_CFLAGS`.
- Omit `-O3` from `COMPILER_RT_TEST_COMPILER_CFLAGS` in non-debug builds for now.
- Provide `__sanitizer::integral_constant<bool, true>::value` instantiation
for `sanitizer_type_traits_test.cpp` in debug builds.
- Disable subtests of `tsan/tests/unit/tsan_trace_test.cpp` that deadlock
in debug builds.
- `XFAIL` `tsan/Linux/check_memcpy.c` in debug builds.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D91620
After https://reviews.llvm.org/rG463aa814182a23 tsan replaces llvm
intrinsics with calls to glibc functions. However this approach is
fragile, as slight changes in pipeline can return llvm intrinsics back.
In particular InstCombine can do that.
Msan/Asan already declare own version of these memory
functions for the similar purpose.
KCSAN, or anything that uses something else than compiler-rt, needs to
implement this callbacks.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D133268
The compler-rt test case tsan/Linux/clone_setns.cpp fails on
PowerPC64 RHEL 7.9 targets.
Unshare fails with errno code EINVAL.
It is unclear why this happens specifically on RHEL 7.9 and no other
operating system like Ubuntu 18 or RHEL 8.4 for example.
This patch uses marcos to disable the test case for ppc64 rhel7.9
because there are no XFAIL directives to target rhel 7.9 specifically.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D130086
An application can use the mere fact of epoll_wait returning an fd
as synchronization with the write on the fd that triggered the notification.
This pattern come up in an internal networking server (b/229276331).
If an fd is added to epoll, setup a link from the fd to the epoll fd
and use it for synchronization as well.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D124518
Turns out the test was working by accident: we need to ensure
TSan instrumentation is not called from the fork() hook, otherwise the
tool will deadlock. Previously it worked because alloc_free_blocks() got
inlined into __tsan_test_only_on_fork(), but it cannot always be the
case.
Adding __attribute__((disable_sanitizer_instrumentation)) will prevent
TSan from instrumenting alloc_free_blocks().
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D120050
Currently the test calls dlclose in the thread
concurrently with the main thread calling a function
from the dynamic library. This is not good.
Wait for the main thread to call the function
before calling dlclose.
Depends on D115612.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D115613
The added test demonstrates loading a dynamic library with static TLS.
Such static TLS is a hack that allows a dynamic library to have faster TLS,
but it can be loaded only iff all threads happened to allocate some excess
of static TLS space for whatever reason. If it's not the case loading fails with:
dlopen: cannot load any more object with static TLS
We used to produce a false positive because dlopen will write into TLS
of all existing threads to initialize/zero TLS region for the loaded library.
And this appears to be racing with initialization of TLS in the thread
since we model a write into the whole static TLS region (we don't what part
of it is currently unused):
WARNING: ThreadSanitizer: data race (pid=2317365)
Write of size 1 at 0x7f1fa9bfcdd7 by main thread:
0 memset
1 init_one_static_tls
2 __pthread_init_static_tls
[[ this is where main calls dlopen ]]
3 main
Previous write of size 8 at 0x7f1fa9bfcdd0 by thread T1:
0 __tsan_tls_initialization
Fix this by ignoring accesses during dlopen.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114953
The test tries to provoke internal allocator to be locked during fork
and then force the child process to use the internal allocator.
This test sometimes deadlocks with the new tsan runtime.
Depends on D114514.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114515
Add a fork test that models what happens on Mac
where fork calls malloc/free inside of our atfork
callbacks.
Reviewed By: vitalybuka, yln
Differential Revision: https://reviews.llvm.org/D114250
Use of gethostent provokes caching of some resources inside of libc.
They are freed in __libc_thread_freeres very late in thread lifetime,
after our ThreadFinish. __libc_thread_freeres calls free which
previously crashed in malloc hooks.
Fix it by setting ignore_interceptors for finished threads,
which in turn prevents malloc hooks.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113989
Start the background thread only after fork, but not after clone.
For fork we did this always and it's known to work (or user code has adopted).
But if we do this for the new clone interceptor some code (sandbox2) fails.
So model we used to do for years and don't start the background thread after clone.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113744
Clone does not exist on Mac.
There are chances it will break on other OSes.
Enable it incrementally starting with Linux only,
other OSes can enable it later as needed.
Reviewed By: melver, thakis
Differential Revision: https://reviews.llvm.org/D113693
When using a static libunwind, the check_memcpy.c can fail because it checks
that tsan intercepted all memcpy/memmoves in the final binary. Though if the
static libunwind is not instrumented, then this will fail because it may contain
regular memcpy/memmoves.
This adds a new REQUIRES check for ensuring that this test won't run unless a
dynamic libunwind.so is provided.
Differential Revision: https://reviews.llvm.org/D111194
These runtime callbacks are supposed to be non-instrumented,
we can't handle runtime recursion well, nor can we afford
explicit recursion checks in the hot functions (memory access,
function entry/exit).
It used to work (not crash), but it won't work with the new runtime.
Mark all runtime callbacks as non-instrumented.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D111157
Tsan's check_memcpy.c test was disabled under debug because it failed.
But it points to real issues and does not help to just disable it.
I tried to enable it and see what fail and the first hit was default ctor for:
struct ChainedOriginDepotDesc {
u32 here_id;
u32 prev_id;
};
initializing these fields to 0's help partially,
but compiler still emits memset before calling ctor.
I did not try to see what's the next failure, because if it fails
on such small structs, it won't be realistic to fix everything
and keep working.
Compile runtimes with -O1 under debug instead.
It seems to fix all current failures. At least I run check-tsan
under clang/gcc x debug/non-debug and all combinations passed.
-O1 does not usually use too aggressive optimizations
and sometimes even makes debugging easier because machine code
is not exceedingly verbose.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D107962
The test contains a race in memset.
The size of reported race depends on how the accessed
memory range split into granules inside of tsan runtime.
The test used to report access of size 8, because presumably
the buffer ended up being aligned to 8 bytes. But after
some unrelated changes this test started to report accesses
of size 1 (presumably .data layout changed), which makes
the test fail.
Guarantee alignment of the buf object explicitly.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107149
The test contains a race in read/write syscalls.
The size of reported race depends on how the accessed
memory range split into granules inside of tsan runtime.
The test used to report access of size 8, because presumably
the buffer ended up being aligned to 8 bytes. But after
some unrelated changes this test started to report accesses
of size 1 (presumably .data layout changed), which makes
the test fail.
Guarantee alignment of the buf object explicitly.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D107131
Add a simple test that uses syscall annotations.
Just to ensure at least basic functionality works.
Also factor out annotated syscall wrappers into a separate
header file as they may be useful for future tests.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D102223
Commit efd254b636 ("tsan: fix deadlock in pthread_atfork callbacks")
fixed another deadlock related to atfork handling.
But builders with DCHECKs enabled reported failures of
pthread_atfork_deadlock2.c and pthread_atfork_deadlock3.c tests
related to the fact that we hold runtime locks on interceptor exit:
https://lab.llvm.org/buildbot/#/builders/70/builds/6727
This issue is somewhat inherent to the current approach,
we indeed execute user code (atfork callbacks) with runtime lock held.
Refactor fork handling to not run user code (atfork callbacks)
with runtime locks held. This change does this by installing
own atfork callbacks during runtime initialization.
Atfork callbacks run in LIFO order, so the expectation is that
our callbacks run last, right before the actual fork.
This way we lock runtime mutexes around fork, but not around
user callbacks.
Extend tests to also install after fork callbacks just to cover
more scenarios. Some tests also started reporting real races
that we previously suppressed.
Also extend tests to cover fork syscall support.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101517
Disable the test on old systems.
pthread_cond_clockwait is supported by glibc-2.30.
It also supported by Android api 30 even though we
do not run tsan on Android.
Fixes https://github.com/google/sanitizers/issues/1259
Reviewed By: dvyukov
Summary:
These tests pass with clang, but fail if gcc was used.
gcc build creates similar but not the same stacks.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Subscribers: dvyukov, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78114
These lit configuration files are really Python source code. Using the
.py file extension helps editors and tools use the correct language
mode. LLVM and Clang already use this convention for lit configuration,
this change simply applies it to all of compiler-rt.
Reviewers: vitalybuka, dberris
Differential Revision: https://reviews.llvm.org/D63658
llvm-svn: 364591
Summary: no_sanitize_thread is not enough as it still puts some tsan instrumentation
Reviewers: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D61393
llvm-svn: 359731
This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:
1) The resulting libFuzzer doesn't expose C++ internals and looks
like a plain C library.
2) We don't have to manually link in libstdc++ to provide cxxabi.
3) The sanitizer tests cannot interfere with an installed version
of libc++.so in LD_LIBRARY_PATH.
Differential Revision: https://reviews.llvm.org/D58013
llvm-svn: 354212
Add pthread_tryjoin_np() and pthread_timedjoin_np() interceptors on Linux,
so that ThreadSanitizer can handle programs using these functions.
Author: Yuri Per (yuri)
Reviewed in: https://reviews.llvm.org/D54521
llvm-svn: 347383
