The Android LLVM build system builds the arm64 fuzzer lib without
HWASan, but then applications that enable HWASan can generated an object
file with a HWASan-ified version of some libc++ symbols (e.g.
`std::__1::piecewise_construct`). The linker can choose the HWASan-ified
definition, but then it cannot resolve the relocation from
libclang_rt.fuzzer-aarch64-android.a to this symbol because the high
bits of the address are unexpectedly set. This produces an error:
```
relocation R_AARCH64_ADR_PREL_PG_HI21 out of range
```
Fix this problem by linking a custom isolated libc++ into Android's
fuzzer library.
We need to pass through ANDROID_NATIVE_API_LEVEL so that the libc++ for
32-bit Android (API < 24) uses LLVM_FORCE_SMALLFILE_FOR_ANDROID.
Instead of a linear scan, use a bitset to track rarity of features. This
improves fuzzer throughput rather dramatically (close to 2x) in early
exploratory phases; in steady state this seems to improve fuzzing
throughput by ~15% according to perf.
The benchmarks are done on an executable with ~100k features, so the
results may change based on the executable that's being fuzzed.
kFeatureSetSize is 2M so the bitset is adding 256 KB to
sizeof(InputCorpus), but this should be fine since there's already three
arrays indexed by feature index for a total of 200 MB.
Switch from C++11 to C++14 as fuzzer requires std::chrono and stdlibc++
doesn't provide chrono literals when using -std=c++11.
Also remove 'u' from ar command to fix this warning: ar: `u' modifier
ignored since `D' is the default (see `U')
Api available since Windows Server 2016/Windows 10 1607
Reviewers: vitalybuka
Reviewed-By: vitalybuka
Differential Revison: https://reviews.llvm.org/D156317
Api available since Windows Server 2016/Windows 10 1607.
Reviewers: vitalybuka
Reviewed-By: vitalybuka
Differential Revision: https://reviews.llvm.org/D156317
Allow to have a name for workers in case the fuzzed code is itself using threads.
Reviewers: vitalybuka
Reviewed-By: vitalybuka
Differential Revision: https://reviews.llvm.org/D155754
Don't hard code the page in FuzzerUtil.h, this breaks on
e.g. LoongArch which defaults to a 16KiB page size.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D140607
This reverts commit a2b677e815.
reverting
Differential Revision: https://reviews.llvm.org/D140607
because <sys/auxv.h> and getauxval() are not available on macOS;
this change is breaking the mac CI bots.
Don't hard code the page in FuzzerUtil.h, this breaks on
e.g. LoongArch which defaults to a 16KiB page size.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D140607
This is an ongoing series of commits that are reformatting our
Python code. This catches the last of the python files to
reformat. Since they where so few I bunched them together.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: jhenderson, #libc, Mordante, sivachandra
Differential Revision: https://reviews.llvm.org/D150784
This fixes building the tests so that the tests can start
executing (even if there still are lots of failures).
Differential Revision: https://reviews.llvm.org/D147648
CopyFileToErr() uses Printf("%s", ...) which fails with a negative size on
files >2Gb (Its path is through var-args wrappers to an unnecessary "%s"
expansion and subject to int overflows) Using puts() in place of printf()
bypasses this path and writes the string directly to stderr. This avoids the
present loss of data when a crashed worker has generated >2Gb of output.
rdar://99384640
Reviewed By: yln, rsundahl
Differential Revision: https://reviews.llvm.org/D146189
There is test for ft file-id in control file, but no test for cov line.
Without the test, a invalid cov file-id would cause crash.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D145672
CopyFileToErr() uses Printf("%s", ...) which fails with a negative size on
files >2Gb (Its path is through var-args wrappers to an unnecessary "%s"
expansion and subject to int overflows) Using puts() in place of printf()
bypasses this path and writes the string directly to stderr. This avoids the
present loss of data when a crashed worker has generated >2Gb of output.
rdar://99384640
Reviewed By: yln
Differential Revision: https://reviews.llvm.org/D146189
/data/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:138:9: error: out-of-line definition of 'Fuzzer' does not match any declaration in 'fuzzer::Fuzzer'
Fuzzer::Fuzzer(UserCallback CB, InputCorpus &Corpus, MutationDispatcher &MD,
^~~~~~
/data/llvm-project/compiler-rt/lib/fuzzer/FuzzerInternal.h:35:10: note: type of 4th parameter of member declaration does not match definition ('fuzzer::FuzzingOptions &' vs 'const fuzzer::FuzzingOptions &')
FuzzingOptions &Options);
^
1 error generated.
`CollectDataFlow()` uses `Flags.collect_data_flow` and
`Flags.data_flow_trace` at the same time. But in the null check before
the invocation, only `Flags.collect_data_flow` is checked, and there is
no other method to make sure `Flags.data_flow_trace` is not null, so
adding a null check for `Flags.data_flow_trace`.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D145040
Some printf format strings in libfuzzer are using the wrong specifizers, fix in this commit.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D145033
When trying to debug some `compiler-rt` unittests, I initially had a hard
time because
- even in a `Debug` build one needs to set `COMPILER_RT_DEBUG` to get
debugging info for some of the code and
- even so the unittests used a hardcoded `-O2` which often makes debugging
impossible.
This patch addresses this by instead using `-O0` if `COMPILER_RT_DEBUG`.
Changes relative to the previous commit:
- Use `string(APPEND)` for `COMPILER_RT_TEST_COMPILER_CFLAGS`.
- Omit `-O3` from `COMPILER_RT_TEST_COMPILER_CFLAGS` in non-debug builds for now.
- Provide `__sanitizer::integral_constant<bool, true>::value` instantiation
for `sanitizer_type_traits_test.cpp` in debug builds.
- Disable subtests of `tsan/tests/unit/tsan_trace_test.cpp` that deadlock
in debug builds.
- `XFAIL` `tsan/Linux/check_memcpy.c` in debug builds.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D91620
When trying to debug some `compiler-rt` unittests, I initially had a hard
time because
- even in a `Debug` build one needs to set `COMPILER_RT_DEBUG` to get
debugging info for some of the code and
- even so the unittests used a hardcoded `-O2` which often makes debugging
impossible.
This patch addresses this by instead using `-O0` if `COMPILER_RT_DEBUG`.
Two tests in `sanitizer_type_traits_test.cpp` need to be disabled since
they have undefined references to `__sanitizer::integral_constant<bool,
true>::value`.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D91620
We currently have an option to select C++ ABI and C++ library for tests
but there are runtimes that use C++ library, specifically ORC and XRay,
which aren't covered by existing options. This change introduces a new
option to control the use of C++ libray for these runtimes.
Ideally, this option should become the default way to select C++ library
for all of compiler-rt replacing the existing options (the C++ ABI
option could remain as a hidden internal option).
Differential Revision: https://reviews.llvm.org/D128036
With this change, fuzz targets may choose to return -1
to indicate that the input should not be added to the corpus
regardless of the coverage it generated.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D128749
We no longer support the use of LLVM_ENABLE_PROJECTS for libcxx and
libcxxabi. We don't use paths to libcxx and libcxxabi in compiler-rt.
Differential Revision: https://reviews.llvm.org/D126905
We no longer support the use of LLVM_ENABLE_PROJECTS for libcxx and
libcxxabi. We don't use paths to libcxx and libcxxabi in compiler-rt.
Differential Revision: https://reviews.llvm.org/D126905
