Kostya Serebryany via llvm-dev
2015-Sep-08 18:12 UTC
[llvm-dev] Some feedback on Libfuzzer
On Sat, Sep 5, 2015 at 11:50 AM, Greg Stark <stark at mit.edu> wrote:> On Sat, Sep 5, 2015 at 6:38 PM, Kostya Serebryany <kcc at google.com> wrote: > > > > This is more like a limitation of asan, not libFuzzer. > > By design, asan does not recover from the first crash. > > This feature has been criticized quite a lot, but I am still convinced > this > > is a feature, not a bug. > > IMHO, recovery mode will be misused/abused too often to be useful, > besides > > it adds complexity to the code. > > (There is a patch under review right now to implement recovery mode for > > asan, > > but I am not sure if or when this patch will be committed) > > Arguably a fuzzer changes the game somewhat. It's one thing to have a > crash during interactive testing but fuzzing is most useful if left to > run on a server unattended. > > However I think this isn't the whole story here. There are many > different kinds of errors and not all are unrecoverable memory > failures. I may just get an internal error from my own code that I > know should never happen but doesn't represent major corruption. I > would like a way for the callback to return an error condition to the > fuzzer driver to have the test case set aside as a crash or perhaps as > a separate category (or perhaps to allow it to specify the name of the > category). >For that you don't need libFuzzer support, right? You can intercept your specific type of bug in the target function.> > >> Also, one more thing, currently Libfuzzer does not catch SIGABRT and > >> treat it as a fatal event. I've added a SIGABRT handler to my own code > >> and moved StaticDeathCallback to public so I can call it from there. > >> > > Again, this is asan, not libFuzzer. > > You need ASAN_OPTIONS=handle_abort=1 > > I hope to make it the default soon-ish. > > Ah, that would have made this work a bit simpler. Thanks. > > I have yet to really experiment with the sanitizers so I don't know if > asan is really doing anything for me given Postgres's internal memory > management.That might be an interesting separate topic to discuss.> I couldn't get msan running but it sounds more promising > to me.If you have custom memory management, msan will be as tricky to use as asan. Also, try ubsan for other kinds of bugs.> I'll have to try again. > > -- > greg >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150908/a095941e/attachment.html>
Sorry, I forgot to check in on this thread. I've made a lot of
progress on my side. I've worked around most of my issues, though
better interfaces to avoid workarounds would be great.
My main problem is the timeouts. Postgres uses ALRM and in fact I want
to specifically invoke that logic to detect cases where it's not
working (I.e. where we don't check for signals for too long). So what
I really want is something independent. I'm actually thinking of
implementing a side process that sends some other signal like SIGQUIT
or SIGFPE periodically.
What I did instead right now is replaced the setitimer with
setrlimit(RLIMIT_CPU) and set the AlarmHandler on SIGXCPU which Linux
fires once a second. That happens to be exactly what I want.
void SetTimer(int Seconds) {
int Res;
struct rlimit limit = {1, RLIM_INFINITY};
Res = setrlimit(RLIMIT_CPU, &limit);
assert(Res == 0);
struct sigaction sigact;
memset(&sigact, 0, sizeof(sigact));
sigact.sa_sigaction = AlarmHandler;
Res = sigaction(SIGXCPU, &sigact, 0);
assert(Res == 0);
}
On 8 Sep 2015 19:12, "Kostya Serebryany" <kcc at google.com>
wrote:>
>
>
> On Sat, Sep 5, 2015 at 11:50 AM, Greg Stark <stark at mit.edu> wrote:
>>
>> However I think this isn't the whole story here. There are many
>> different kinds of errors and not all are unrecoverable memory
>> failures. I may just get an internal error from my own code that I
>> know should never happen but doesn't represent major corruption. I
>> would like a way for the callback to return an error condition to the
>> fuzzer driver to have the test case set aside as a crash or perhaps as
>> a separate category (or perhaps to allow it to specify the name of the
>> category).
>
>
> For that you don't need libFuzzer support, right?
> You can intercept your specific type of bug in the target function.
That's exactly what I'm doing. I'm catching internal errors now and
call abort() so the fuzzer logs that test case as a crash.
What I'm doing now is (man I wish gmail was more useful for code):
diff --git a/FuzzerLoop.cpp b/FuzzerLoop.cpp
index dd81616..aa53046 100644
--- a/FuzzerLoop.cpp
+++ b/FuzzerLoop.cpp
@@ -77,6 +77,28 @@ void Fuzzer::AlarmCallback() {
}
}
+void Fuzzer::StaticErrorCallback(const char *errorname) {
+ assert(F);
+ F->ErrorCallback(errorname);
+}
+
+#include <string.h>
+void Fuzzer::ErrorCallback(const char *ErrorName) {
+ char prefix[80];
+ assert(strnlen(ErrorName, 80) < 80);
+ if (!ErrorName) {
+ strcpy(prefix, "error-");
+ } else {
+ sprintf(prefix, "%s-", ErrorName);
+ }
+
+ Printf("ERROR: Fuzzer Function reports an internal error\n");
+ if (CurrentUnit.size() <= kMaxUnitSizeToPrint)
+ Print(CurrentUnit, "\n");
+ PrintUnitInASCIIOrTokens(CurrentUnit, "\n");
+ WriteUnitToFileWithPrefix(CurrentUnit, prefix);
+}
So now my test function can call StaticErrorCallback("myerror") and
the driver will log that test in a file named myerror-<hash> and
return control to me. I can then return normally and the fuzzer will
continue. (Sorry for the C string.h)
It may be that I should just not use the Libfuzzer driver at all. Just
use the various functions from Libfuzzer for generating and keeping
track of cases but keep the main loop under my code.
>> I have yet to really experiment with the sanitizers so I don't know
if
>> asan is really doing anything for me given Postgres's internal
memory
>> management.
>
> That might be an interesting separate topic to discuss.
Fwiw I found it easy to hook ASAN_POISON macros using the existing
valgrind support so that's all good now.
> If you have custom memory management, msan will be as tricky to use as
asan.
> Also, try ubsan for other kinds of bugs.
I think msan might be possible to hook with the same macros but I
can't get a working binary out of it. I get
configure:4042: ./conftest
FATAL: Code 0x5634731ac3f0 is out of application range. Non-PIE build?
FATAL: MemorySanitizer can not mmap the shadow memory.
FATAL: Make sure to compile with -fPIE and to link with -pie.
FATAL: Disabling ASLR is known to cause this error.
FATAL: If running under GDB, try 'set disable-randomization off'.
I get that even if I put -fPIE in CFLAGS.
On Sat, Sep 12, 2015 at 7:48 PM, Greg Stark <stark at mit.edu> wrote:> I get that even if I put -fPIE in CFLAGS.Er, yeah. Even a trivial test case doesn't work: $ cat foo.c int main(int argc, char *argv[], char *envp[]) { return 1; } $ clang -o foo -fsanitize=memory -fPIE -pie foo.c $ sysctl kernel.randomize_va_space kernel.randomize_va_space = 2 $ ./foo FATAL: Code 0x55873d194390 is out of application range. Non-PIE build? FATAL: MemorySanitizer can not mmap the shadow memory. FATAL: Make sure to compile with -fPIE and to link with -pie. FATAL: Disabling ASLR is known to cause this error. FATAL: If running under GDB, try 'set disable-randomization off'. ==25950==Process memory map follows: 0x55873d177000-0x55873d216000 /tmp/foo 0x55873d415000-0x55873d419000 /tmp/foo 0x55873d419000-0x55873f88c000 0x7f276d5cf000-0x7f276d921000 0x7f276d921000-0x7f276dac0000 /lib/x86_64-linux-gnu/libc-2.19.so 0x7f276dac0000-0x7f276dcc0000 /lib/x86_64-linux-gnu/libc-2.19.so 0x7f276dcc0000-0x7f276dcc4000 /lib/x86_64-linux-gnu/libc-2.19.so 0x7f276dcc4000-0x7f276dcc6000 /lib/x86_64-linux-gnu/libc-2.19.so 0x7f276dcc6000-0x7f276dcca000 0x7f276dcca000-0x7f276dce0000 /lib/x86_64-linux-gnu/libgcc_s.so.1 0x7f276dce0000-0x7f276dedf000 /lib/x86_64-linux-gnu/libgcc_s.so.1 0x7f276dedf000-0x7f276dee0000 /lib/x86_64-linux-gnu/libgcc_s.so.1 0x7f276dee0000-0x7f276dee3000 /lib/x86_64-linux-gnu/libdl-2.19.so 0x7f276dee3000-0x7f276e0e2000 /lib/x86_64-linux-gnu/libdl-2.19.so 0x7f276e0e2000-0x7f276e0e3000 /lib/x86_64-linux-gnu/libdl-2.19.so 0x7f276e0e3000-0x7f276e0e4000 /lib/x86_64-linux-gnu/libdl-2.19.so 0x7f276e0e4000-0x7f276e1e4000 /lib/x86_64-linux-gnu/libm-2.19.so 0x7f276e1e4000-0x7f276e3e3000 /lib/x86_64-linux-gnu/libm-2.19.so 0x7f276e3e3000-0x7f276e3e4000 /lib/x86_64-linux-gnu/libm-2.19.so 0x7f276e3e4000-0x7f276e3e5000 /lib/x86_64-linux-gnu/libm-2.19.so 0x7f276e3e5000-0x7f276e3ec000 /lib/x86_64-linux-gnu/librt-2.19.so 0x7f276e3ec000-0x7f276e5eb000 /lib/x86_64-linux-gnu/librt-2.19.so 0x7f276e5eb000-0x7f276e5ec000 /lib/x86_64-linux-gnu/librt-2.19.so 0x7f276e5ec000-0x7f276e5ed000 /lib/x86_64-linux-gnu/librt-2.19.so 0x7f276e5ed000-0x7f276e605000 /lib/x86_64-linux-gnu/libpthread-2.19.so 0x7f276e605000-0x7f276e804000 /lib/x86_64-linux-gnu/libpthread-2.19.so 0x7f276e804000-0x7f276e805000 /lib/x86_64-linux-gnu/libpthread-2.19.so 0x7f276e805000-0x7f276e806000 /lib/x86_64-linux-gnu/libpthread-2.19.so 0x7f276e806000-0x7f276e80a000 0x7f276e80a000-0x7f276e82a000 /lib/x86_64-linux-gnu/ld-2.19.so 0x7f276ea03000-0x7f276ea08000 0x7f276ea1e000-0x7f276ea2a000 0x7f276ea2a000-0x7f276ea2b000 /lib/x86_64-linux-gnu/ld-2.19.so 0x7f276ea2b000-0x7f276ea2c000 /lib/x86_64-linux-gnu/ld-2.19.so 0x7f276ea2c000-0x7f276ea2d000 0x7ffd99d31000-0x7ffd99d52000 [stack] 0x7ffd99d73000-0x7ffd99d75000 [vvar] 0x7ffd99d75000-0x7ffd99d77000 [vdso] 0xffffffffff600000-0xffffffffff601000 [vsyscall] ==25950==End of process memory map. -- greg