This fixes a nit I had accidentally introduced in
https://github.com/llvm/llvm-project/pull/85142
I don't think the value of __msan_get_track_origins() will change
between the start and end of InitShadowWithReExec, but it's cleaner to
use the parameter.
Since this standalone build configuration uses the runtime libraries that
are being built just now, we need to ensure that e.g. the TSan unit tests
depend on the tsan runtime library. Also fix TSAN_DEPS being overridden
to not include the tsan runtime (commit .....).
This change fixes a build race seen in the CI checks for
TsanRtlTest-x86_64-Test in https://github.com/llvm/llvm-project/pull/83088.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83650
With https://github.com/llvm/llvm-project/pull/83088, we now need the
runtimes to be built before running test if
COMPILER_RT_TEST_STANDALONE_BUILD_LIBS is true, since otherwise we
get failures running `ninja check-all` such as the following:
```
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.fuzzer-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-basic-x86_64.a: No such file or directory
/usr/bin/ld: cannot find .../compiler-rt/cmake-build-all-sanitizers/lib/linux/libclang_rt.xray-fdr-x86_64.a: No such file or directory
```
This is a follow-up to 058e9b03 which started removing these checks
and it should make it easier to stop forcing COMPILER_RT_STANDALONE_BUILD
for runtimes builds in the future.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83651
This ports the change from TSan
(0784b1eefa).
Testing notes: run 'sudo sysctl vm.mmap_rnd_bits=32; ninja check-msan'
before and after this patch.
N.B. aggressive ASLR may also cause the app to overlap with the
allocator region; for MSan, this was fixed in
af2bf86a37
MSan divides the virtual address space into APP, INVALID, SHADOW and
ORIGIN memory. The allocator usually just steals a bit of the APP
address space: typically the bottom portion of the PIE binaries section,
which works because the Linux kernel maps from the top of the PIE
binaries section. However, if ASLR is very aggressive, the binary may
end up mapped in the same location where the allocator wants to live;
this results in a segfault.
This patch adds in a MappingDesc::ALLOCATOR type and enforces that the
memory range for the allocator is not occupied by anything else.
Since the allocator range information is not readily available in
msan.h, we duplicate the information from msan_allocator.cpp.
Note: aggressive ASLR can also lead to a different type of failure,
where the PIE binaries/libraries are mapped entirely outside of the
APP/ALLOCATOR sections; that will be addressed in a separate patch
(https://github.com/llvm/llvm-project/pull/85142).
All these unit tests already include ${COMPILER_RT_GTEST_SOURCE} as an
input source file and the target llvm_gtest does not exist for
standalone builds. Currently the DEPS argument is ignored for standalone
builds so the missing target is not a problem, but as part of fixing a
build race for standalone builds I am planning to include those
dependencies in COMPILER_RT_TEST_STANDALONE_BUILD_LIBS configurations.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83649
Most sanitizers don't support static linking. One primary reason is the
incompatibility with interceptors. `GetTlsSize` is another reason.
asan/memprof use `__interception::DoesNotSupportStaticLinking`
(`_DYNAMIC` reference) to reject -static at link time. Port this
detector to other sanitizers. dfsan actually supports -static for
certain cases. Don't touch dfsan.
This ensures that the MSan unit tests are able to pass with an
uninstrumented libunwind. We need to avoid instrumentation for
__gxx_personality_v0, which is part of the default msan_ignorelist.txt
that is installed into the resource directory. However, if we are trying
to test the just-built libraries, this global ignore list may not be
present yet, so we still instrument the function.
Arguably this function should not be on the default ignore list since it
is only a problem when building libcxxabi with MSan instrumentation and
without an instrumented libunwind, so maybe the logic should really be
part of the libcxxabi build. However, that could be done as a follow-up.
See 2f856a36e0 for more context.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83652
This is preparation for performance optimization.
We need to highlight that this is very specific lock, and should not be
used for other purposes.
Add `fork_child` parameter to distinguish processes after fork.
Introduce a new virtual class StackTracePrinter and an implementation
FormattedStackTracePrinter in preparation of enabling symbolizer markup
for linux.
This change allows us to implement other behaviour under the same api
for StackTracePrinter, for example, MarkupStackTracePrinter.
Reason for revert: A missing header file for the
sanitizer_symbolizer_markup.cpp files.
This was not caught in local builds or pre-merge checks given that to
trigger the error, the code
has to be compiled for Fuchsia.
For this reland I've build for the fuchsia targets as well as linux.
Introduce a new virtual class StackTracePrinter and an implementation
FormattedStackTracePrinter in preparation of enabling symbolizer markup
for linux.
This change allows us to implement other behaviour under the same api
for StackTracePrinter, for example, MarkupStackTracePrinter.
s390x is one of the architectures where the "long double" type was changed
from a 64-bit IEEE to a 128-bit IEEE type back in the glibc 2.4 days.
This means that glibc still exports two versions of the long double functions
(those that already existed back then), and we have to intercept the correct
version. There is already an existing define SANITIZER_NLDBL_VERSION that
indicates this situation, we simply have to respect it when intercepting
strtold and wcstold.
In addition, on s390x a long double return value is passed in memory via
implicit reference. This means the interceptor for functions returning
long double has to unpoison that memory slot, or else we will get
false-positive uninitialized memory reference warnings when the caller
accesses that return value - similar to what is already done in the
mallinfo interceptor. Create a variant macro INTERCEPTOR_STRTO_SRET and
use it on s390x.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D159378
SANITIZER_GLIBC is always defined so should be tested with an if not an
ifdef.
Fixes: ad7e250100
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D159041
This removes and replaces usage of a few LowLevelAllocators with a single one
provided by sanitizer_common. Functionally, there should be no difference
between using different allocators vs the same one. This works really well
with D158783 which controls the size of each allocator mmap to significantly
reduce fragmentation.
This doesn't remove them all, mainly the ones used by asan and the flag parser.
Differential Revision: https://reviews.llvm.org/D158786
`strtol("0b1", 0, 0)` can be (pre-C23) 0 or (C23) 1.
`sscanf("0b10", "%i", &x)` is similar. glibc 2.38 introduced
`__isoc23_strtol` and `__isoc23_scanf` family functions for binary
compatibility.
When `_ISOC2X_SOURCE` is defined (implied by `_GNU_SOURCE`) or
`__STDC_VERSION__ > 201710L`, `__GLIBC_USE_ISOC2X` is defined to 1 and
these `__isoc23_*` symbols are used.
Add `__isoc23_` versions for the following interceptors:
* sanitizer_common_interceptors.inc implements strtoimax/strtoumax.
Remove incorrect FIXME about https://github.com/google/sanitizers/issues/321
* asan_interceptors.cpp implements just strtol and strtoll. The default
`replace_str` mode checks `nptr` is readable and `endptr` is writable.
atoi reuses the existing strtol interceptor.
* msan_interceptors.cpp implements strtol family functions and their
`_l` versions. Tested by lib/msan/tests/msan_test.cpp
* sanitizer_common_interceptors.inc implements scanf family functions.
The strtol family functions are spreaded, which is not great, but the
patch (intended for release/17.x) does not attempt to address the issue.
Add symbols to lib/sanitizer_common/symbolizer/scripts/global_symbols.txt to
support both glibc pre-2.38 and 2.38.
When build bots migrate to glibc 2.38+, we will lose test coverage for
non-isoc23 versions since the existing C++ unittests imply `_GNU_SOURCE`.
Add test/sanitizer_common/TestCases/{strtol.c,scanf.c}.
They catch msan false positive in the absence of the interceptors.
Fix https://github.com/llvm/llvm-project/issues/64388
Fix https://github.com/llvm/llvm-project/issues/64946
Link: https://lists.gnu.org/archive/html/info-gnu/2023-07/msg00010.html
("The GNU C Library version 2.38 is now available")
Reviewed By: #sanitizers, vitalybuka, mgorny
Differential Revision: https://reviews.llvm.org/D158943
Currently if a program calls sigaction very early (before non-lazy sanitizer
initialization, in particular if .preinit_array initialization is not enabled),
then sigaction will wrongly fail since the interceptor is not initialized yet.
In all other interceptors we do lazy runtime initialization for this reason,
but we don't do it in the signal interceptors.
Do lazy runtime initialization in signal interceptors as well.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D155188
Relanding with #if SANITIZER_GLIBC to avoid breaking FreeBSD.
Also incorporates Arthur's BUILD.gn fix (thanks!) from https://reviews.llvm.org/rGc1e283851772ba494113311405d48cfb883751d1
Original commit message:
This patch adds an msan interceptor for dladdr1 (with support for RTLD_DL_LINKMAP and RTLD_DL_SYMENT) and an accompanying test. It also adds a helper file, msan_dl.cpp, that contains UnpoisonDllAddrInfo (refactored out of the dladdr interceptor) and UnpoisonDllAddr1ExtraInfo.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D154272
Reland with -Wcast-qual issue fixed
Original commit message:
This patch adds an msan interceptor for dladdr1 (with support for RTLD_DL_LINKMAP and RTLD_DL_SYMENT) and an accompanying test. It also adds a helper file, msan_dl.cpp, that contains UnpoisonDllAddrInfo (refactored out of the dladdr interceptor) and UnpoisonDllAddr1ExtraInfo.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D154272
This patch adds an msan interceptor for dladdr1 (with support
for RTLD_DL_LINKMAP and RTLD_DL_SYMENT) and an accompanying
test. It also adds a helper file, msan_dl.cpp, that contains
UnpoisonDllAddrInfo (refactored out of the dladdr interceptor)
and UnpoisonDllAddr1ExtraInfo.
Differential Revision: https://reviews.llvm.org/D154272
This patch adds basic memory sanitizer support for loongarch64
with 47-bit VMA, which memory layout is based on x86_64.
The LLVM part of the LoongArch memory sanitizer implementation will
be done separately, which will fix failing tests in check-msan.
These failing tests fail with the following same error: "error in
backend: unsupported architecture".
Reviewed By: #sanitizers, vitalybuka, MaskRay
Differential Revision: https://reviews.llvm.org/D140528
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
v4:
- Add interface attribute to __sanitizer_internal_mem* declarations as
well, as otherwise some compilers (MSVC) will complain.
- Add SANITIZER_COMMON_NO_REDEFINE_BUILTINS to source files using
C++STL, since this could lead to ODR violations (see added comment).
v3:
- Don't use ALIAS() to alias internal_mem*() functions to
__sanitizer_internal_mem*() functions, but just define them as
ALWAYS_INLINE functions instead. This will work on darwin and windows.
v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
v3:
- Don't use ALIAS() to alias internal_mem*() functions to
__sanitizer_internal_mem*() functions, but just define them as
ALWAYS_INLINE functions instead. This will work on darwin and windows.
v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).
In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.
To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).
In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.
Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:
1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.
2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.
The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D151152
This moves memintrinsic interceptors (memcpy/memmove/memset) into a new
file sanitizer_common_interceptors_memintrinsics.inc.
This is in preparation of redefining builtins, however, we must be
careful to not redefine builtins in TUs that define interceptors of the
same name.
In all cases except for MSan, memintrinsic interceptors were moved to a
new TU $tool_interceptors_memintrinsics.cpp. In the case of MSan, it
turns out this is not yet necessary (as shown by the later patch
introducing memcpy tests).
NFC.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D151552
