N3322 makes NULL + 0 well-defined in C, matching the C++ semantics.
Adjust the pointer-overflow sanitizer to no longer report NULL + 0 as a
pointer overflow in any language mode. NULL + nonzero will of course
continue to be reported.
As N3322 is part of
https://www.open-std.org/jtc1/sc22/wg14/www/previous.html, and we never
performed any optimizations based on NULL + 0 being undefined in the
first place, I'm applying this change to all C versions.
The gcov version is set to 11.1 (compatible with gcov 9) even if
`-Xclang -coverage-version=` specified version is less than 11.1.
Therefore, we can drop producer support for version < 11.1.
This patch disables the `noasan-strcmp.test` for AArch64 Linux as it
consistently fails on the buildbot machine while passing on other
AArch64 Linux systems.
We have seen similar issues on noasan-strncmp.test in past which had
random failures on certain machines/environments.
Following buildbot is failing in both check stage1 and stage2:
https://lab.llvm.org/buildbot/#/builders/121/builds/711
Based on #115987, with the introduction of `REQUIRES: continuous-mode`.
Also Linux assumes `runtime_reloc`.
FIXME: image-with-no-counters.c is still excluded.
Linking this runtime requires C++ ABI, which breaks -nostdlib++ builds.
However, UBSAN C++ runtime is only needed for CFI and VPTR checks.
Unblocks #120370.
1. -f[no-]sanitize-link-c++-runtime suppose to
override defauld behavior implied from `CCCIsCXX`
2. Take into account -nostdlib++ (unblocks #108357)
3. Fix typo hasFlag vs hasArg.
This patch introduces the runtime components for type sanitizer: a
sanitizer for type-based aliasing violations.
It is based on Hal Finkel's https://reviews.llvm.org/D32197.
C/C++ have type-based aliasing rules, and LLVM's optimizer can exploit
these given TBAA metadata added by Clang. Roughly, a pointer of given
type cannot be used to access an object of a different type (with, of
course, certain exceptions). Unfortunately, there's a lot of code in the
wild that violates these rules (e.g. for type punning), and such code
often must be built with -fno-strict-aliasing. Performance is often
sacrificed as a result. Part of the problem is the difficulty of finding
TBAA violations. Hopefully, this sanitizer will help.
For each TBAA type-access descriptor, encoded in LLVM's IR using
metadata, the corresponding instrumentation pass generates descriptor
tables. Thus, for each type (and access descriptor), we have a unique
pointer representation. Excepting anonymous-namespace types, these
tables are comdat, so the pointer values should be unique across the
program. The descriptors refer to other descriptors to form a type
aliasing tree (just like LLVM's TBAA metadata does). The instrumentation
handles the "fast path" (where the types match exactly and no
partial-overlaps are detected), and defers to the runtime to handle all
of the more-complicated cases. The runtime, of course, is also
responsible for reporting errors when those are detected.
The runtime uses essentially the same shadow memory region as tsan, and
we use 8 bytes of shadow memory, the size of the pointer to the type
descriptor, for every byte of accessed data in the program. The value 0
is used to represent an unknown type. The value -1 is used to represent
an interior byte (a byte that is part of a type, but not the first
byte). The instrumentation first checks for an exact match between the
type of the current access and the type for that address recorded in the
shadow memory. If it matches, it then checks the shadow for the
remainder of the bytes in the type to make sure that they're all -1. If
not, we call the runtime. If the exact match fails, we next check if the
value is 0 (i.e. unknown). If it is, then we check the shadow for the
remainder of the byes in the type (to make sure they're all 0). If
they're not, we call the runtime. We then set the shadow for the access
address and set the shadow for the remaining bytes in the type to -1
(i.e. marking them as interior bytes). If the type indicated by the
shadow memory for the access address is neither an exact match nor 0, we
call the runtime.
The instrumentation pass inserts calls to the memset intrinsic to set
the memory updated by memset, memcpy, and memmove, as well as
allocas/byval (and for lifetime.start/end) to reset the shadow memory to
reflect that the type is now unknown. The runtime intercepts memset,
memcpy, etc. to perform the same function for the library calls.
The runtime essentially repeats these checks, but uses the full TBAA
algorithm, just as the compiler does, to determine when two types are
permitted to alias. In a situation where access overlap has occurred and
aliasing is not permitted, an error is generated.
As a note, this implementation does not use the compressed shadow-memory
scheme discussed previously
(http://lists.llvm.org/pipermail/llvm-dev/2017-April/111766.html). That
scheme would not handle the struct-path (i.e. structure offset)
information that our TBAA represents. I expect we'll want to further
work on compressing the shadow-memory representation, but I think it
makes sense to do that as follow-up work.
This includes build fixes for Linux from Mingjie Xu.
Depends on #76260 (Clang support), #76259 (LLVM support)
PR: https://github.com/llvm/llvm-project/pull/76261
**Related:** #117925
**About this PR:**
This PR performs 3 small but related fixes for ASan users on Windows:
1. It ensures that the `allocator_may_return_null` flag is honored when
set through the user function `__asan_default_options`. For more
details, please see: #117925
2. It adds a missing `AllocatorMayReturnNull()` check inside
`InternalAlloc` that's needed to avoid error'ing out when the allocator
_correctly_ returns `null` when `allocator_may_return_null` is set.
3. In `sanitizer_win`'s `ReturnNullptrOnOOMOrDie`, it allows returning
`null` when the last error is set to `ERROR_INVALID_PARAMETER` which may
be set by `VirtualAlloc` on WIndows when attempting to allocate
exceedingly large memory.
I've added test cases that should cover these new behaviors. Happy to
take on any feedback as well. Thank you :-)
---------
Co-authored-by: David Justo <dajusto@microsoft.com>
Add the `dump_at_exit` flag to control whether or not profiles should be
dumped when the program exits. Since we can call
`__memprof_profile_dump()` directly, we don't necessarily need to dump
profiles at exit.
The functions are not relevant for most sanitizers and only required for
MSan to see which regions have been written to. This eliminates a link
dependency for all other sanitizers and fixes#59007: while `-lresolv`
had been added for the static runtime in 6dce56b2a3, it wasn't added
to the shared runtimes.
Instead of just moving the interceptors, we adapt them to MSan
conventions:
* We don't skip intercepting when `msan_init_is_running` is true, but
directly call ENSURE_MSAN_INITED() like most other interceptors. It
seems unlikely that these functions are called during initialization.
* We don't unpoison `errno`, because none of the functions is specified
to use it.
This re-applies 570ecdcf8b, which was reverted in 74e8a37ff3 due to bot
failures. This commit renames sysv_resolve.cpp to resolve.cpp, which was the
cause of the config errors.
This reapplies 570ecdcf8b, which was reverted in 6073dd923b due to bot
failures.
The test failures on Linux were fixed by:
1. Removing an overly restrictive assertion (query dependence on a symbol no
longer implies a MaterializingInfo for that symbol)
2. Adding reentry and resolver files to the ORC runtime CMakeLists.txt for
Linux.
3. Adding the __orc_rt_reentry -> __orc_rt_sysv_reentry alias to ELFNixPlatform.
…d reentry.
These utilities provide new, more generic and easier to use support for
lazy compilation in ORC.
LazyReexportsManager is an alternative to LazyCallThroughManager. It
takes requests for lazy re-entry points in the form of an alias map:
lazy-reexports = {
( <entry point symbol #1>, <implementation symbol #1> ),
( <entry point symbol #2>, <implementation symbol #2> ),
...
( <entry point symbol #n>, <implementation symbol #n> )
}
LazyReexportsManager then:
1. binds the entry points to the implementation names in an internal
table.
2. creates a JIT re-entry trampoline for each entry point.
3. creates a redirectable symbol for each of the entry point name and
binds redirectable symbol to the corresponding reentry trampoline.
When an entry point symbol is first called at runtime (which may be on
any thread of the JIT'd program) it will re-enter the JIT via the
trampoline and trigger a lookup for the implementation symbol stored in
LazyReexportsManager's internal table. When the lookup completes the
entry point symbol will be updated (via the RedirectableSymbolManager)
to point at the implementation symbol, and execution will proceed to the
implementation symbol.
Actual construction of the re-entry trampolines and redirectable symbols
is delegated to an EmitTrampolines functor and the
RedirectableSymbolsManager respectively.
JITLinkReentryTrampolines.h provides a JITLink-based implementation of
the EmitTrampolines functor. (AArch64 only in this patch, but other
architectures will be added in the near future).
Register state save and reentry functionality is added to the ORC
runtime in the __orc_rt_sysv_resolve and __orc_rt_resolve_implementation
functions (the latter is generic, the former will need custom
implementations for each ABI and architecture to be supported, however
this should be much less effort than the existing OrcABISupport
approach, since the ORC runtime allows this code to be written as native
assembly).
The resulting system:
1. Works equally well for in-process and out-of-process JIT'd code.
2. Requires less boilerplate to set up.
Given an ObjectLinkingLayer and PlatformJD (JITDylib containing the ORC
runtime), setup is just:
```c++
auto RSMgr = JITLinkRedirectableSymbolManager::Create(OLL);
if (!RSMgr)
return RSMgr.takeError();
auto LRMgr = createJITLinkLazyReexportsManager(OLL, **RSMgr, PlatformJD);
if (!LRMgr)
return LRMgr.takeError();
```
after which lazy reexports can be introduced with:
```c++
JD.define(lazyReexports(LRMgr, <alias map>));
```
LazyObectLinkingLayer is updated to use this new method, but the LLVM-IR
level CompileOnDemandLayer will continue to use LazyCallThroughManager
and OrcABISupport until the new system supports a wider range of
architectures and ABIs.
The llvm-jitlink utility's -lazy option now uses the new scheme. Since
it depends on the ORC runtime, the lazy-link.ll testcase and associated
helpers are moved to the ORC runtime.
Add the `__memprof_default_options_str` variable, initialized via the
`-memprof-runtime-default-options` LLVM flag, to hold the default options string
for memprof. This allows us to set these options during compile time in
the clang invocation.
Also update the docs to describe the various ways to set these options.
We don't need 128-bit integers here, instead rewrite the fp_test.h helpers
to avoid the need for __int128. Also change the high argument for
compareResultF80() and F80FromRep80() to be uint16_t since values greater
than this do not make any sense. This should allow the compiler to detect
accidentally swapping the arguments.
Reviewed By: biabbas, alexander-shaposhnikov
Pull Request: https://github.com/llvm/llvm-project/pull/117818
This is a complex one - `syscall` is used when people want to bypass
libc and make the call directly
However, this call:
* Has a variable amount of arguments (up to 6, typically)
* Has arguments that can be any type, (think of whatever arguments go in
to the libc call, or see more details here
https://syscalls.mebeim.net/?table=x86/64/x64/latest)
I've tried to put in a couple tests to ensure we aren't mucking with the
underlying functionality and they seem to be working.
Similar to `__asan_default_options`, users can specify default options
upon building the instrumented binaries by providing their own
definition of `__xray_default_options` which returns the option strings.
This is useful in cases where setting the `XRAY_OPTIONS` environment
variable might be difficult. Plus, it's a convenient way to populate
XRay options when you always want the instrumentation to be enabled.
Using a `--section-start` address in the test was causing link errors on
some targets. Now it uses a linker script to move the note after `.bss`,
which should still have the kind of memory offset we're looking for.
This is a follow-up to #114907 to fix buildbot errors.
This relands #86209 which was reverted because ./bin/llvm no longer
accepted test paths in the source tree instead of the build tree. This was
happening because `add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit`
was called before all tsst suites were registered, and therefore it was
missing the source->build dir mappings.
Original commit message:
I am currently trying to test the LLVM runtimes (including compiler-rt)
against an installed LLVM tree rather than a build tree (since that is
no longer available). Currently, the runtimes build of compiler-rt assumes
that LLVM_BINARY_DIR is writable since it uses configure_file() to write
there during the CMake configure stage. Instead, generate this file inside
CMAKE_CURRENT_BINARY_DIR, which will match LLVM_BINARY_DIR when invoked
from llvm/runtimes/CMakeLists.txt.
I also needed to make a minor change to the hwasan tests: hwasan_symbolize
was previously found in the LLVM_BINARY_DIR, but since it is generated as
part of the compiler-rt build it is now inside the CMake build directory
instead. I fixed this by passing the output directory to lit as
config.compiler_rt_bindir and using llvm_config.add_tool_substitutions().
For testing that we no longer write to the LLVM install directory as
part of testing or configuration, I created a read-only bind mount and
configured the runtimes builds as follows:
```
$ sudo mount --bind --read-only ~/llvm-install /tmp/upstream-llvm-readonly
$ cmake -DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_COMPILER=/tmp/upstream-llvm-readonly/bin/clang \
-DCMAKE_CXX_COMPILER=/tmp/upstream-llvm-readonly/bin/clang++ \
-DLLVM_INCLUDE_TESTS=TRUE -DLLVM_ENABLE_ASSERTIONS=TRUE \
-DCOMPILER_RT_INCLUDE_TESTS=TRUE -DCOMPILER_RT_DEBUG=OFF \
-DLLVM_ENABLE_RUNTIMES=compiler-rt \
-DCMAKE_DISABLE_FIND_PACKAGE_LLVM=TRUE \
-DCMAKE_DISABLE_FIND_PACKAGE_Clang=TRUE \
-G Ninja -S ~/upstream-llvm-project/runtimes \
-B ~/upstream-llvm-project/runtimes/cmake-build-debug-llvm-git
```
Pull Request: https://github.com/llvm/llvm-project/pull/114307
I've run into an issue where TSan can't be used on some code without
turning off deadlock detection because a thread tries to hold too many
mutexes. It would be preferable to be able to use deadlock detection as
that is a major benefit of TSan.
Its mentioned in https://github.com/google/sanitizers/issues/950 that
the 64 mutex limit was an arbitrary number. I've increased it to 128 and
all the tests still pass. Considering the increasing number of cores on
CPUs and how programs can now use more threads to take advantage of it,
I think raising the limit to 128 would be some good future proofing
---------
Co-authored-by: Vitaly Buka <vitalybuka@google.com>
This function is always examining its own ELF headers in memory, but it
was trying to use conditions between examining files or memory, and it
wasn't accounting for LOAD offsets at runtime. This is especially bad if
a loaded segment has additional padding that's not in the file offsets.
Now we do a first scan of the program headers to figure out the runtime
base address based on `PT_PHDR` and/or `PT_DYNAMIC` (else assume zero),
similar to libc's `do_start`. Then each `PT_NOTE` pointer is simply the
base plus the segments's `pt_vaddr`, which includes LOAD offsets.
Fixes#114605
Following the example of tsan, where we took the name
This would allow users to determine if they want to see ALL output from
rtsan.
Additionally, remove the UNLIKELY hint, as it is now up to the flag whether or
not it is likely that we go through this conditional.
This tests that a simple C++ static initializer works as expected.
Compared to the architecture specific, assembly level regression tests for the
ORC runtime; this test is expected to catch cases where the compiler adopts
some new MachO feature that the ORC runtime does not yet support (e.g. a new
initializer section).
Adding support for the extra SUMMARY line that is output by most
compilers. This also adds the ability for end-users to specify their own
handlers for reporting these errors (see the test).
This patch implements support for `-fxray-shared` on AArch64 and fixes a
remaining issue in the previous PR #114431.
A bug in the XRay `CMakeLists.txt` caused the XRay assembly sources to
be built for every architecture in `XRAY_DSO_SUPPORTED_ARCH` on Apple.
This led to the compiler trying to compile AArch64 assembly for X86
targets and vice versa.
This is addressed here by ensuring that assembly sources are only built
for the matching architecture (see fixup commit).
**Original PR description:**
This patch adds support for `-fxray-shared` on AArch64. This feature,
introduced in #113548 for x86_64, enables the instrumentation of shared
libraries with XRay.
Changes:
- Adds AArch64 to the list of targets supporting `-fxray-shared`
- Introduces PIC versions of the AArch64 XRay trampolines
- Adjusts relevant XRay tests
