Commit Graph

78 Commits

Author SHA1 Message Date
Matt Morehouse
ddf352b953 [libFuzzer] Include TEMP_MAX_LEN in Fuzzer::PrintStats.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43597

llvm-svn: 325817
2018-02-22 19:00:17 +00:00
Matt Morehouse
36c89b3c77 [libFuzzer] Set -experimental_len_control=1000 as default.
Summary:
Experiments using
https://github.com/google/fuzzer-test-suite/tree/master/engine-comparison
show a significant increase in coverage and reduction in corpus size
with this option enabled.

Addresses https://llvm.org/pr36371.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D42932

llvm-svn: 325050
2018-02-13 20:52:15 +00:00
Petr Hosek
2f70693e08 [Fuzzer] Use Zircon's public API on Fuchsia
The original libFuzzer Fuchsia port relied on convenience libraries,
but these are not exported as part of Fuchsia sysroot. This change
eliminates the use of these libraries and relies on public API only.

Differential Revision: https://reviews.llvm.org/D42996

llvm-svn: 324454
2018-02-07 08:22:58 +00:00
Jonas Hahnfeld
44ef345c50 [CMake] Remove -stdlib= which is unused when passing -nostdinc++
This avoids the warnings when building with LLVM_ENABLE_LIBCXX
which automatically adds -stdlib=libc++ to CMAKE_CXX_FLAGS.

Differential Revision: https://reviews.llvm.org/D42238

llvm-svn: 323969
2018-02-01 13:57:24 +00:00
Petr Hosek
b1eaa56704 [fuzzer] Update and enable libFuzzer on Fuchsia
This change updates the Fuchsia-specific code to use the C++ friendly
duration expressions and flips on the building of
libclang_rt.fuzzer-x86_64.a and similar for Fuchsia. Given that
compiler-rt doesn't build on Fuchsia, test have been run by explicitly
building the library and linking it against
lib/fuzzer/tests/FuzzerUnittest.cpp.

Differential Revision: https://reviews.llvm.org/D42670

llvm-svn: 323828
2018-01-30 22:59:48 +00:00
Matt Morehouse
1d8e5ea250 [libFuzzer] Fix sizeof(ptr) bug.
sizeof(const char *) returns 4 or 8 when what we really want is the size
of the array.

llvm-svn: 323802
2018-01-30 18:25:55 +00:00
Petr Hosek
066e4bf888 Reland "[Fuzzer] Parametrize add_custom_libcxx"
add_custom_libcxx uses the just built compiler and installs the
built libc++, e.g. for testing, neither of which is desirable in
case of Fuzzer where the libc++ should be built using the host
compiler and it's only linked into the libFuzzer and should never
be installed. This change introduces additional arguments to
add_custom_libcxx to allow parametrizing its behavior.

Differential Revision: https://reviews.llvm.org/D42330

llvm-svn: 323054
2018-01-21 01:01:53 +00:00
Petr Hosek
81ac12d1b4 Revert "[Fuzzer] Parametrize add_custom_libcxx"
This reverts commit r323032: failing on the sanitizer-x86_64-linux-autoconf bot.

llvm-svn: 323033
2018-01-20 09:21:00 +00:00
Petr Hosek
94e67be187 [Fuzzer] Parametrize add_custom_libcxx
add_custom_libcxx uses the just built compiler and installs the
built libc++, e.g. for testing, neither of which is desirable in
case of Fuzzer where the libc++ should be built using the host
compiler and it's only linked into the libFuzzer and should never
be installed. This change introduces additional arguments to
add_custom_libcxx to allow parametrizing its behavior.

Differential Revision: https://reviews.llvm.org/D42330

llvm-svn: 323032
2018-01-20 09:03:16 +00:00
Petr Hosek
eac2b47b9f Reland "[libFuzzer] Support using libc++"
This is needed in case the users of libFuzzer use libc++ in their
code, which the fuzz target (libFuzzer) will be linked against.
When libc++ source is available, we build a private version of it
and link it against libFuzzer which allows using the same static
library against codebases which use both libc++ and libstdc++.

Differential Revision: https://reviews.llvm.org/D37631

llvm-svn: 322755
2018-01-17 20:39:14 +00:00
Petr Hosek
68bc4bd6e5 Revert "[libFuzzer] Support using libc++"
This reverts commit r322604: test is failing for standalone compiler-rt.

llvm-svn: 322689
2018-01-17 17:24:56 +00:00
Petr Hosek
a1b57e694e [libFuzzer] Support using libc++
This is needed in case the users of libFuzzer use libc++ in their
code, which the fuzz target (libFuzzer) will be linked against.
When libc++ source is available, we build a private version of it
and link it against libFuzzer which allows using the same static
library against codebases which use both libc++ and libstdc++.

Differential Revision: https://reviews.llvm.org/D37631

llvm-svn: 322604
2018-01-17 00:42:48 +00:00
Kamil Rytarowski
4463ae4f6d Switch from Bourne shell to simply base shell to build libfuzzer
Summary: It is not necessary launching the build script with bash.

Reviewers: krytarowski

Reviewed By: krytarowski

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D42008

llvm-svn: 322422
2018-01-12 23:45:24 +00:00
Kamil Rytarowski
e81e944199 lib Fuzzer FreeBSD support
Summary: Patch by David CARLIER

Reviewers: vitalybuka, kcc, dim, emaste, davide, morehouse, george.karpenkov

Reviewed By: morehouse

Subscribers: george.karpenkov, kubamracek, srhines, mgorny, emaste, krytarowski

Differential Revision: https://reviews.llvm.org/D41642

llvm-svn: 322380
2018-01-12 17:15:05 +00:00
Max Moroz
a40ce7bc36 [libfuzzer] Fix UB when calculating Log(0) in StackDepthStepFunction().
Summary:
__builtin_clz used for Log calculation returns an undefined result
when argument is 0. I noticed that issue when was testing some fuzzers:

```
/src/libfuzzer/FuzzerTracePC.h:282:33: runtime error: shift exponent 450349 is too large for 32-bit type 'uint32_t' (aka 'unsigned int')
  #0 0x43d83f in operator() /src/libfuzzer/FuzzerTracePC.h:283:33
  #1 0x43d83f in void fuzzer::TracePC::CollectFeatures<fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*)::$_1>(fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*)::$_1) const /src/libfuzzer/FuzzerTracePC.h:290
  #2 0x43cbd4 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/libfuzzer/FuzzerLoop.cpp:445:7
  #3 0x43e5f1 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:706:5
  #4 0x43e9e1 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:739:3
  #5 0x432f8c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:754:6
  #6 0x42ee18 in main /src/libfuzzer/FuzzerMain.cpp:20:10
  #7 0x7f17ffeb182f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
  #8 0x407838 in _start (/out/rotate_fuzzer+0x407838)

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41457

llvm-svn: 321211
2017-12-20 19:31:51 +00:00
Matt Morehouse
6e294952b6 [libFuzzer] Add dummy call of LLVMFuzzerTestOneInput to afl_driver.
Summary:
Add dummy call of LLVMFuzzerTestOneInput to afl_driver before it starts
executing on actual inputs. Do this so that first time initialization
performed by LLVMFuzzerTestOneInput is not considered code covered by
a particular input.

Patch By: metzman

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: llvm-commits, Sanitizers

Differential Revision: https://reviews.llvm.org/D41193

llvm-svn: 320643
2017-12-13 22:02:44 +00:00
Kostya Serebryany
e9ed2327b6 [libFuzzer] change the strategy for -experimental_len_control to grow max_len slower
llvm-svn: 320531
2017-12-12 23:11:28 +00:00
Kostya Serebryany
de891a1946 [libFuzzer] even less aggressive step function for stack depth. Also don't use asan in one test to speed it up
llvm-svn: 320259
2017-12-09 19:18:10 +00:00
Kostya Serebryany
f94d87aadf [libFuzzer] make the stack depth signal less agressive, otherwise we are sometimes exploding the corpus size. This still needs more analysis and tuning
llvm-svn: 320237
2017-12-09 02:45:23 +00:00
Matt Morehouse
ffb1132e52 [libFuzzer] Add support for Fuchsia OS.
Summary:
This patch adds the initial support for Fuchsia.

- LIBFUZZER_FUCHSIA is added as an OS type in FuzzerDefs.h
- Fuchsia is, by design, not POSIX compliant. However, it does use ELF and
  supports common POSIX I/O functions. Thus, FuzzerExtFunctions.h and
  FuzzerIO.h are implemented by extending the header guards in
  FuzzerExtFunctionsWeak.cpp and FuzzerIOPosix.cpp to include
  LIBFUZZER_FUCHSIA.
- The platform-specific portions of FuzzerUtil.h are implemented by
  FuzzerUtilFuchsia.cpp, which makes use of exception ports, syscalls, and
  the launchpad library.
- The experimental equivalence server is not currently supported, so
  FuzzerShmem.h is implemented by stub methods in FuzzerShmemFuchsia.cpp.
  Any future implementation will likely involve VMOs.

Tested with ASAN/SanCov on Fuchsia/x86-64 with the canonical toy fuzzer.

Patch By: aarongreen

Reviewers: kcc, morehouse, flowerhack, phosek

Reviewed By: kcc, phosek, Eugene.Zelenko

Subscribers: srhines, mgorny, Eugene.Zelenko

Differential Revision: https://reviews.llvm.org/D40974

llvm-svn: 320210
2017-12-08 22:54:44 +00:00
Kostya Serebryany
2659c63e2e [libFuzzer] honor -use_counters, sligntly change the meaning of -experimental_len_control, call UpdateFeatureFrequency only if instructed by the flag
llvm-svn: 320205
2017-12-08 22:21:42 +00:00
Kostya Serebryany
a97bd9a7d3 [libFuzzer] Decrease stack usage in unit tests
Summary: With 3 Dictionary objects, each containing space of ~16k DictionaryEntry objects, the MutationDispatcher object is fairly memory heavy.  On platforms with a lower default stack size, this can cause panics in FuzzerUnittest as those tests stack-allocate the MutationDispatcher.  This may be especially problematic for platforms that do not (yet) have a way to programmatically change their stack size, aside from link-time flags.  In general, it seems more prudent to use the heap for an object of this size.

Reviewers: kcc, morehouse

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D40926

llvm-svn: 319988
2017-12-06 23:35:02 +00:00
Kostya Serebryany
7ac58ee3e1 [libFuzzer] fix a minor regression in printing
llvm-svn: 319975
2017-12-06 22:12:24 +00:00
Matt Morehouse
056774d13b [libFuzzer] Make redirects happen in proper sequence.
"> file" must come before "2>&1" to have redirection occur correctly in
all cases.  Fixes a regression on minimize_two_crashes.test.

llvm-svn: 319792
2017-12-05 17:13:17 +00:00
Matt Morehouse
e911a5d356 [libFuzzer] Remove const from ignoreRemainingArgs return value.
In this case const does nothing but trigger a warning.

llvm-svn: 319685
2017-12-04 20:06:52 +00:00
Matt Morehouse
dc81fd6cec [libFuzzer] Remove FuzzerCommand.cpp from build sources.
FuzzerCommand.cpp was not introduced in r319680.  Instead, it was
implemented in the header file.

llvm-svn: 319682
2017-12-04 19:54:14 +00:00
Matt Morehouse
04304d129b [libFuzzer] Encapsulate commands in a class.
Summary:
To be more portable (especially w.r.t. platforms without system()),
commands should be managed programmatically rather than via string
manipulation on the command line. This change introduces
Fuzzer::Command, with methods to manage arguments and flags, set output
options, and execute the command.

Patch By: aarongreen

Reviewers: kcc, morehouse

Reviewed By: kcc, morehouse

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D40103

llvm-svn: 319680
2017-12-04 19:25:59 +00:00
Kostya Serebryany
de9bafb162 [libFuzzer] add a flag -malloc_limit_mb
llvm-svn: 319590
2017-12-01 22:12:04 +00:00
Kostya Serebryany
6afa7a54be [libFuzzer] remove stale flags; NFC
llvm-svn: 319572
2017-12-01 19:24:06 +00:00
Kostya Serebryany
ad05ee0512 [libFuzzer] add an experimental search heuristic flag -reduce_depth
llvm-svn: 319571
2017-12-01 19:18:38 +00:00
Dan Liew
37516b527b [LibFuzzer] Improve comments on CounterToFeature() function.
This is based on discussion in https://reviews.llvm.org/D40376 .

The comments try to explain the reason for the current implementation
and note that it might change in the future, so clients should not
rely on this particular implementation.

Differential Revision: https://reviews.llvm.org/D40565

llvm-svn: 319190
2017-11-28 17:41:58 +00:00
Vitaly Buka
065421f0cc [fuzzer] Initialize PcDescr buffer before calling __sanitizer_symbolize_pc
__sanitizer_symbolize_pc is not instrumented so msan assumes that PcDescr
was not initialized.

llvm-svn: 318488
2017-11-17 00:46:59 +00:00
Kostya Serebryany
cb74c65f5e libfuzzer: Fix file listing on some filesystems
Summary:
For some filesystems, readdir will not populate dirent::d_type with valuable information. This causes libfuzzer to proceed with an empty corpus, instead of the file it contains.

This has been tested on a server using XFS.

It should fix https://bugs.llvm.org//show_bug.cgi?id=25991

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D40028

llvm-svn: 318303
2017-11-15 16:45:17 +00:00
Kostya Serebryany
8eb9ca2580 [libFuzzer] respect max_len during merge
llvm-svn: 318302
2017-11-15 16:42:52 +00:00
Kostya Serebryany
54a8456005 [libFuzzer] make sure to flush IO when done merging one file
llvm-svn: 317835
2017-11-09 21:30:33 +00:00
Matt Morehouse
947838c9c5 [libFuzzer] Don't add leaking inputs to corpus.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39850

llvm-svn: 317831
2017-11-09 20:44:08 +00:00
Kostya Serebryany
a2ca2dcc46 [libFuzzer] handle SIGUSR1/SIGUSR2 and try to exit grafully on these signals
llvm-svn: 317829
2017-11-09 20:30:19 +00:00
Kostya Serebryany
3ac31d4453 [libFuzzer] allow merge to resume after being preempted
llvm-svn: 317767
2017-11-09 05:49:28 +00:00
Kostya Serebryany
68fdef1f7a [libFuzzer] allow user to specify the merge control file
llvm-svn: 317747
2017-11-09 01:05:29 +00:00
Vitaly Buka
7d22324b87 [fuzzer] Fix nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317186
2017-11-02 04:12:10 +00:00
Vitaly Buka
ca9075272b [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317119
2017-11-01 20:27:06 +00:00
Vitaly Buka
7dbc1d8433 [fuzzer] Fix threaded stack printing
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317071
2017-11-01 03:02:59 +00:00
Vitaly Buka
df00e89cdc Revert "[fuzzer] Fix threaded stack printing and nested mallocs"
Fails on darwin

Revert "[fuzzer] Script to detect unbalanced allocation in -trace_malloc output"
Needs previous one.

This reverts commit r317034, r317036.

llvm-svn: 317061
2017-11-01 00:55:52 +00:00
Vitaly Buka
0747700ce1 [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317036
2017-10-31 20:50:07 +00:00
Vitaly Buka
58da33e35c [fuzzer] Fix threaded stack printing and nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317034
2017-10-31 20:49:48 +00:00
Alex Shlyapnikov
5ded0701a9 [Sanitizers-libFuzzer] Addressing coding style issues.
Summary: The result of clang-format and few manual changes (as prompted on D39155).

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39211

llvm-svn: 316395
2017-10-23 23:24:33 +00:00
Alex Shlyapnikov
6f1c26f222 [libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing sessions.
Summary:
Fuzzing targets that allocate/deallocate a lot of memory tend to consume
a lot of RSS when ASan quarantine is enabled. Purging quarantine between
iterations and returning memory to OS keeps RSS down and should not
reduce the quarantine effectiveness provided the fuzz target does not
preserve state between iterations (in this case this feature can be turned off).

Based on D39153.

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39155

llvm-svn: 316382
2017-10-23 22:04:30 +00:00
Kostya Serebryany
fc3079963f [libFuzzer] print a better warning if we hit the ld bug
llvm-svn: 315778
2017-10-14 00:07:11 +00:00
Alex Shlyapnikov
29d3ba7576 [Sanitizers] Always build libFuzzer with frame pointers.
Lacking frame pointers, sanitized fuzzers collect bogus stack
traces and the set of stack traces grows indefinitely, leading to OOMs.

llvm-svn: 315770
2017-10-13 21:57:43 +00:00
Kostya Serebryany
51823d3aae [libFuzzer] tweam use_feature_frequency to be less aggressive; run a dummy input before the seed corpus
llvm-svn: 315657
2017-10-13 01:12:23 +00:00