When building with debug info enabled, some load/store instructions do
not have a DebugLocation attached. When using the default IRBuilder, it
attempts to copy the DebugLocation from the insertion-point instruction.
When there's no DebugLocation, no attempt is made to add one.
Add a fallback DebugLocation with the help of InstrumentationIRBuilder for
memintrinsics. In particular, the compiler may optimize load/store without
debug info into memintrinsics, which then are missing debug info as well.
When building the kernel with LTO, KCOV & debug information enabled,
multiple inlinable SanitizerCoverage functions require debug information
present.
In such cases we repurpose the InstrumentationIRBuilder that ensures
the necessary debug information is added if necessary.
This has been done analogous to the work for the ThreadSanitizer
in D124937.
Bug: https://github.com/ClangBuiltLinux/linux/issues/1721
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D155377
When building the kernel with LTO, KASAN & debug information enabled,
multiple inlinable AddressSanitizer functions require debug information
present.
In such cases we repurpose the InstrumentationIRBuilder that ensures
the necessary debug information is added if necessary.
This has been done analogous to the work for the ThreadSanitizer
in D124937.
Bug: https://github.com/ClangBuiltLinux/linux/issues/1721
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D155376
There's no caller to `promoteIndirectCalls` that would pass a nullptr
`ModuleAnalysisManager`, so passing it by reference does away with a
bunch of nullptr tests, and also removes the need for a "OwnedORE".
Differential Revision: https://reviews.llvm.org/D155027
This restores commit b4a82b6225, reverted
in 3ab7ef28ee because it was thought to
cause a bot failure, which ended up being unrelated to this patch set.
Differential Revision: https://reviews.llvm.org/D154856
This restores commit 29252fdd53, reverted
in 3498cf52ba because it was thought to
cause a bot failure, which ended up being unrelated to this patch set.
Differential Revision: https://reviews.llvm.org/D154872
The patch uses a way similiar to vp.load/store and consider the mask popcount as
the effetive vector length.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D151713
Previously the MemProf profile was expected to be in the same profile
file as a normal PGO profile, passed via the usual -fprofile-use=
option, and was matched in the same pass. To simplify profile
preparation, since the raw MemProf profile requires the binary for
symbolization and may be simpler to index separately from the raw PGO
profile, and also to enable providing a MemProf profile for a SamplePGO
build, separate out the MemProf feedback option and matching pass.
This patch adds the -fmemory-profile-use=${file} option, and the
provided file is passed down to LLVM and ultimately used in a new
MemProfUsePass which performs the matching of just the memory profile
contents of that file.
Note that a single profile file containing both normal PGO and MemProf
profile data is still supported, and the relevant profile data is
matched by the appropriate matching pass(es) based on which option(s)
the profile is provided with (the same profile file can be supplied to
both feedback options).
Differential Revision: https://reviews.llvm.org/D154856
Split out of D154856, this prepares for the addition of a new dedicated
memory profile matching pass.
Differential Revision: https://reviews.llvm.org/D154872
The patch handle masked.gather/scatter just like the way D149245 handles
vp.gather/scatter.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D151545
Make minor changes to enable DFSAN and its tests on
loongarch64. And port Linux loongarch64 memory mappings
from msan.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D140690
This patch enabled msan in LLVM and fixed all failing tests in
check-msan.
It does not add VarArgHelper implementation on LoongArch, which
will be done separately later. And it adds a test for VarArgNoOpHelper,
which is based on the X86 one.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D152692
Partial progress towards removing in-tree uses of `Type::getPointerTo`,
before we can deprecate the API.
If the API is used solely to support an unnecessary bitcast, get rid of
the bitcast as well.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D153933
Move `AttributeMask` out of `llvm/IR/Attributes.h` to a new file
`llvm/IR/AttributeMask.h`. After doing this we can remove the
`#include <bitset>` and `#include <set>` directives from `Attributes.h`.
Since there are many headers including `Attributes.h`, but not needing
the definition of `AttributeMask`, this causes unnecessary bloating of
the translation units and slows down compilation.
This commit adds in the include directive for `llvm/IR/AttributeMask.h`
to the handful of source files that need to see the definition.
This reduces the total number of preprocessing tokens across the LLVM
source files in lib from (roughly) 1,917,509,187 to 1,902,982,273 - a
reduction of ~0.76%. This should result in a small improvement in
compilation time.
Differential Revision: https://reviews.llvm.org/D153728
When a 128-bit load/store is aligned by 8, we incorrectly emit `load i16, ptr ..., align 2`
while the shadow memory address may not be aligned by 2.
This manifests as possibly-misaligned shadow memory load with `-mstrict-align`,
e.g. `clang --target=aarch64-linux -O2 -mstrict-align -fsanitize=address`
```
__attribute__((noinline)) void foo(unsigned long *ptr) {
ptr[0] = 3;
ptr[1] = 3;
}
// ldrh w8, [x9, x8] // the shadow memory load may not be aligned by 2
```
Infer the shadow memory alignment from the load/store alignment to set the
correct alignment. The generated code now uses two ldrb and one orr.
Fix https://github.com/llvm/llvm-project/issues/63258
Differential Revision: https://reviews.llvm.org/D152663
ARM stores the Thumb state in the least significant bit of the
function pointers. When compiling for ARM or Thumb, as all
instructions are at least 16-bit aligned, ignore the LSB when
computing the prefix hash location, so we can support both
pure Thumb and mixed ARM/Thumb binaries.
Fixes#62936
Reviewed By: MaskRay, simon_tatham
Differential Revision: https://reviews.llvm.org/D152484
KCFI traps should always be recoverable, but as Intrinsic::trap
is marked noreturn, it's not possible to continue execution after
handling the trap as the compiler is free to assume we never
return. Switch to debugtrap instead to ensure we have the option
to resume execution after the trap.
This patch makes the variables names for callback functions more consistent. Changes no functionality.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D151605
This patch implements `__hwasan_memset_match_all`, `__hwasan_memcpy_match_all` and `__hwasan_memmove_match_all`, making hwasan-match-all-tag flag working for hwasan versions of memset, memcpy and memmove.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D149943
Currently, hwasan-match-all-tag flag is supported in inline memory access instrumentation and outline memory access instrumentation, but not supported in callback memory access instrumentation.
- For inline memory access instrumentation: a hwasan-match-all-tag check is added following the tag-mismtach check, if tag from pointer is mismatched with tag from shadow memory and tag from pointer is not equal with hwasan-match-all-tag, then a tag-mismatch will be report.
- For outline memory acess instrumentation: MatchAllTag is encoded in AccessInfo, when emit HWASAN memaccess symbols, asm-printer emits assembly instructions to check if tag from pointer is equal with hwasan-match-all-tag.
- For callback memory access instrumentation: hwasan-match-all-tag check is not implemented in `__hwasan_load`/`__hwasan_store`.
This patch implements a set of callback functions: `__hwasan_[load|store][1|2|4|8|16|n]_match_all` and `__hwasan_load[load|store][1|2|4|8|16|n]_match_all_noabort`, making hwasan-match-all-tag flag working for callback memory access instrumentation.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D149580
The patch supports vp.gather/scatter by allowing addresses being pointer vectors.
And then we just need to check each active pointer element of those pointer vectors.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D149245
This patch adds nosantize metadata to memory access instructions inserted by gcov emitProfileNotes(), making sanitizers skip these instructions when gcov and sanitizer are used together.
Reviewed By: nickdesaulniers
Differential Revision: https://reviews.llvm.org/D150460
The method is marked for deprecation. Delete the method and move all of
its consumers to use the DomTreeUpdater version.
Reviewed By: foad
Differential Revision: https://reviews.llvm.org/D149428
This patch adds a new method setNoSanitizeMetadata() for Instruction, and use it in SanitizerMetadata and SanitizerCoverage.
Reviewed By: nickdesaulniers, MaskRay
Differential Revision: https://reviews.llvm.org/D150632
The patch adds new member MaybeStride into InterestingMemoryOperand to represent
the stride value of experimental.vp.strided.load/store.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D149073
Tag selection for global variables is sequential, starting at a
pseduo-ish seed that's based on the hash of the file name.
Previously, it was possible for a global to be assigned a tag in the
range [1,15]. If the global's size was not a multiple of granules (i.e.
`size % 16 != 0`), then the last granule of the global would be assigned
a short granule tag as well.
If the real memory tag of the global (e.g. '04') happened to collide
with the short granule tag (e.g. '04'), then __hwasan_check would
see that the memory tag matched the short granule tag, and dutifully
assume (in this fast check) that everthing is okay.
Unfortunately, if you tried to access the [5,15]th byte, you never get
to the short granule check. This means you miss intra-granule overflows
on the last granule of a global, if said global was assigned a real
memory tag in the range [1,15].
This causes flakiness in certain global tests, if the SHA of the
filename changes between runs.
This problem also exists for heap and stack allocations as well, but
there's a concern that if we exclude the [1,15] range for heap and stack
that it's an unhappy tradeoff. On Android, this would mean that a 1/255
chance of false positive becomes 1/240. On other platforms though (that
have a less-than-8-bit tag space), this may be unacceptable. We can
think about potential fixes for that in other places, but globals are
fine to reduce the space, as really the only thing that matters is
catching sequential overflow. The false-negative scenario is much, much
more common in use-after-free and use-after-scope, which globals don't
have.
Differential Revision: https://reviews.llvm.org/D150742
Annotation metadata supports adding singular annotation strings to annotation block. This patch adds the ability to insert a tuple of strings into the metadata array.
The idea here is that each tuple of strings represents a piece of information that can be all related. It makes it easier to parse through related metadata information given it will be contained in one tuple.
For example in remarks any pass that implements annotation remarks can have different type of remarks and pass additional information for each.
The original behaviour of annotation remarks is preserved here and we can mix tuple annotations and single annotations for the same instruction.
Reviewed By: paquette
Differential Revision: https://reviews.llvm.org/D148328
The patch adds new member MaybeEVL into InterestingMemoryOperand to represent
the effective vector length for vp intrinsics. It may be extended for some target intrinsics in the future.
Reviewed By: kito-cheng
Differential Revision: https://reviews.llvm.org/D146208
Now that the runtime tracks the lifetime access density directly, we can
use that directly in the threshold checks instead of less accurately
computing from other statistics.
Differential Revision: https://reviews.llvm.org/D149684
This commit removes potential UB in the PGO instrumentation passes that
was caused by casting away constness and then potentially modifying the
object.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D148903
This is folloup to b5595836, which missed the
Replacemen variable.
Before b5595836 the code assumed that alloca
ptrs are not tagged so tagging is implemented
as simple OR.
So this patch completes support of tagged SP
by passing untagged alloca pointers into
tagPointer.
c55fffe replaced fcmp with fpclass.
```
declare i1 @llvm.is.fpclass(<fptype> <op>, i32 <test>)
declare <N x i1> @llvm.is.fpclass(<vector-fptype> <op>, i32 <test>)
```
Perfect fix will require checking bits of <op> corresponding to <test>
argument. For now just propagate shadow without reporting before
intrinsic. Still existing handling of fcmp is also simple OR, so it's
not making it worse.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D149491
This commit cleans up some parts of the PGO instrumentation. Most
importantly, it removes a template parameter shadowing of a class name
that could lead to confusion.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D149324
This commit moves the CFGMST.h file into the include directory. The
implemented algorithm is can be helpful for downstream projects that
want to use the PGO data in a non-standard way.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D149336
Enable -fsanitize=kernel-memory support in Clang.
The x86_64 ABI requires that shadow_origin_ptr_t must be returned via a
register pair, and the s390x ABI requires that it must be returned via
memory pointed to by a hidden parameter. Normally Clang takes care of
the ABI, but the sanitizers run long after it, so unfortunately they
have to duplicate the ABI logic.
Therefore add a special case for SystemZ and manually emit the
s390x-ABI-compliant calling sequences. Since it's only 2 architectures,
do not create a VarArgHelper-like abstraction layer.
The kernel functions are compiled with the "packed-stack" and
"use-soft-float" attributes. For the "packed-stack" functions, it's not
correct for copyRegSaveArea() to copy 160 bytes of shadow and origins,
since the save area is dynamically sized. Things are greatly simplified
by the fact that the vararg "use-soft-float" functions use precisely
56 bytes in order to save the argument registers to where va_arg() can
find them.
Make copyRegSaveArea() copy only 56 bytes in the "use-soft-float" case.
The "packed-stack" && !"use-soft-float" case has no practical uses at
the moment, so leave it for the future.
Add tests.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D148596
