llvm dev - Aug 2017 - how to auto-report LLVM bugs found by fuzzing?

On Tue, Aug 29, 2017 at 4:13 PM, Justin Bogner <mail at justinbogner.com>
wrote:
> Kostya Serebryany <kcc at google.com> writes:
> > Hi,
> >
> > We have several llvm fuzz targets running on OSS-Fuzz, a continuous
> > automated fuzzing service:
> > https://github.com/google/oss-fuzz
> > https://www.usenix.org/sites/default/files/conference/protected-files/
> usenixsecurity17_slides_serebryany.pdf
> >
> > It has reported a few bugs in cxa_demangler, clang, and dwarfdump
> already,
> > and we expect to add more fuzz targets to it soon (llvm-isel-fuzzer,
> > clang-format-fuzzer, ...)
> >
> > A question to everyone: how do we report these bugs properly?
> > OSS-Fuzz files bugs automatically into a separate bug tracker, it can
not
> > file bugs to bugzilla.
> > By default, the bug reports are private for security reasons, and only
> > those CC-ed explicitly can see them.
> >
> > Should we make the bug reports public by default?
> > We can set things differently for the llvm project (llvm, clang, etc)
> and
> > libcxxabi (demangler):
> > https://github.com/google/oss-fuzz/tree/master/projects/llvm
> > https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi
>
> At least some of these should probably just be public by default. Things
> like llvm-isel-fuzzer or clang-fuzzer aren't really looking for
security
> bugs, so I wouldn't expect them to find stuff that falls under the
> responsible disclosure umbrella.
>
So, how about making all LLVM bugs public by default and leaving
cxa_demangler bugs private?
(I can't make it finer-grained, see below)

>
> This should be thought about on a case by case basis, of course.
>
> > Should we automatically CC the bugs to any of the llvm maliing lists
> (e.g.
> > llvm-dev)?
>
> Perhaps we could CC them to llvm-bugs? That's the same list that new
> bugzilla bugs are announced to.
>
Ah, good idea.
Unless someone objects I'll add llvm-bugs to the spam^W list :)

>
> > If a bug is CC-ed to a list, everyone will see the bug report summary
in
> > e-mail,
> > but if the bug remains private the reproducer for the bug will remain
> > private.
> >
> > Who wants to be CC-ed explicitly?
> > (please add yourself to
> > https://github.com/google/oss-fuzz/blob/master/projects/
> llvm/project.yaml)
>
> Can this be set up to CC per-fuzz-target or so? I'm sure some people
are
> interested in, say, clang, but not necessarily cxa_demangler, or
> vice-versa.
>
Sadly, no.

We can distinguish llvm_cxxabi (cxa_demangler) from everything else because
these are currently two independent projects on oss-fuzz.
Making it finer-grained would require setting up separate oss-fuzz projects
which is harder to maintain and would not be welcome on oss-fuzz side.
The automatic e-mails announce the fuzz target's name, so filters will be
easy to set up.

>
> > Examples of bug reports follow.
> >
> > Thanks!
> >
> > --kcc
> >
> >
> > dwarfdump:
> >
> > https://bugs.chromium.org/p/oss-fuzz/issues/detail?id>
3216&q=label%3AProj-llvm%20dwarfdump-fuzzer&colspec=ID%
> 20Type%20Component%20Status%20Proj%20Reported%20Owner%20Summary
> >
> > Crash Type: ASSERT
> > Crash Address:
> > Crash State:
> >   result <= UINT32_MAX
> >   llvm::object::WasmObjectFile::parseStartSection
> >   llvm::object::WasmObjectFile::parseSection
> >
> > Crash Type: Heap-buffer-overflow READ 1
> > Crash Address: 0x60200000009a
> > Crash State:
> >   llvm::object::WasmObjectFile::parseCustomSection
> >   llvm::object::WasmObjectFile::parseSection
> >   llvm::object::WasmObjectFile::WasmObjectFile
> >
> > Crash Type: Heap-buffer-overflow READ 1
> > Crash Address: 0x604000000776
> > Crash State:
> >   llvm::StringMapImpl::LookupBucketFor
> >   std::pair<llvm::StringMapIterator<unsigned int>, bool>
> > llvm::StringMap<unsigned
> >   llvm::DWARFContext::create
> >
> > Crash Type: Heap-buffer-overflow READ 4
> > Crash Address: 0x60300000011c
> > Crash State:
> >   llvm::identify_magic
> >   llvm::object::ObjectFile::createObjectFile
> >   _start
> >
> > clang-fuzzer:
> > https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q>
label%3AProj-llvm+clang-fuzzer&colspec=ID+Type+Component+Status+Proj+
> Reported+Owner+Summary&cells=ids
> > Crash Type: Stack-buffer-overflow READ 1
> > Crash Address: 0x7f79e7b71760
> > Crash State:
> >   clang::Lexer::SkipLineComment
> >   clang::Lexer::LexTokenInternal
> >   clang::Lexer::Lex
> >
> > Crash Type: Direct-leak
> > Crash Address:
> > Crash State:
> >   clang::Parser::ParseParameterDeclarationClause
> >   clang::Parser::ParseFunctionDeclarator
> >   clang::Parser::ParseDirectDeclarator
> >
> >
> > Crash Type: Stack-overflow
> > Crash Address: 0x7ffc78d69f48
> > Crash State:
> >   clang::StmtVisitorBase<clang::make_const_ptr, IntExprEvaluator,
> > bool>::Visit
> >   Evaluate
> >   IntExprEvaluator::VisitBinaryOperator
> >
> > Crash Type: ASSERT
> > Crash Address:
> > Crash State:
> >   !Prev.isAmbiguous() && "Cannot have an ambiguity in
> previous-declaration
> > lookup"
> >   DiagnoseInvalidRedeclaration
> >   clang::Sema::ActOnFunctionDeclarator
> >
> >
> > cxa_demangler:
> >
> > https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q>
label%3AProj-llvm_libcxxabi&colspec=ID+Type+Component+
> Status+Proj+Reported+Owner+Summary&cells=ids
> >
> >
> > Crash Type: Heap-buffer-overflow READ 8
> > Crash Address: 0x619000000078
> > Crash State:
> >   __cxxabiv1::parse_encoding
> >   __cxxabiv1::demangle
> >   __cxa_demangle
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170829/3f4305aa/attachment.html>

Kostya Serebryany <kcc at google.com> writes:
> On Tue, Aug 29, 2017 at 4:13 PM, Justin Bogner <mail at
justinbogner.com>
> wrote:
>
>> Kostya Serebryany <kcc at google.com> writes:
>> > Hi,
>> >
>> > We have several llvm fuzz targets running on OSS-Fuzz, a
continuous
>> > automated fuzzing service:
>> > https://github.com/google/oss-fuzz
>> >
https://www.usenix.org/sites/default/files/conference/protected-files/
>> usenixsecurity17_slides_serebryany.pdf
>> >
>> > It has reported a few bugs in cxa_demangler, clang, and dwarfdump
>> already,
>> > and we expect to add more fuzz targets to it soon
(llvm-isel-fuzzer,
>> > clang-format-fuzzer, ...)
>> >
>> > A question to everyone: how do we report these bugs properly?
>> > OSS-Fuzz files bugs automatically into a separate bug tracker, it
can not
>> > file bugs to bugzilla.
>> > By default, the bug reports are private for security reasons, and
only
>> > those CC-ed explicitly can see them.
>> >
>> > Should we make the bug reports public by default?
>> > We can set things differently for the llvm project (llvm, clang,
etc)
>> and
>> > libcxxabi (demangler):
>> > https://github.com/google/oss-fuzz/tree/master/projects/llvm
>> >
https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi
>>
>> At least some of these should probably just be public by default.
Things
>> like llvm-isel-fuzzer or clang-fuzzer aren't really looking for
security
>> bugs, so I wouldn't expect them to find stuff that falls under the
>> responsible disclosure umbrella.
>>
>
> So, how about making all LLVM bugs public by default and leaving
> cxa_demangler bugs private?
> (I can't make it finer-grained, see below)
This sounds good to me.
>>
>> This should be thought about on a case by case basis, of course.
>>
>> > Should we automatically CC the bugs to any of the llvm maliing
lists
>> (e.g.
>> > llvm-dev)?
>>
>> Perhaps we could CC them to llvm-bugs? That's the same list that
new
>> bugzilla bugs are announced to.
>>
>
> Ah, good idea.
> Unless someone objects I'll add llvm-bugs to the spam^W list :)
>
>>
>> > If a bug is CC-ed to a list, everyone will see the bug report
summary in
>> > e-mail,
>> > but if the bug remains private the reproducer for the bug will
remain
>> > private.
>> >
>> > Who wants to be CC-ed explicitly?
>> > (please add yourself to
>> > https://github.com/google/oss-fuzz/blob/master/projects/
>> llvm/project.yaml)
>>
>> Can this be set up to CC per-fuzz-target or so? I'm sure some
people are
>> interested in, say, clang, but not necessarily cxa_demangler, or
>> vice-versa.
>>
>
> Sadly, no.
>
> We can distinguish llvm_cxxabi (cxa_demangler) from everything else because
> these are currently two independent projects on oss-fuzz.
> Making it finer-grained would require setting up separate oss-fuzz projects
> which is harder to maintain and would not be welcome on oss-fuzz side.
> The automatic e-mails announce the fuzz target's name, so filters will
be
> easy to set up.
Fair enough.

Bugs found by oss-fuzz in llvm are now public:
https://bugs.chromium.org/p/oss-fuzz/issues/list?q=proj-llvm
(and the new ones will be public too).
I've also added llvm-bugs at lists.llvm.org to the list of e-mail
recipients:
https://github.com/google/oss-fuzz/blob/master/projects/llvm/project.yaml

On Tue, Aug 29, 2017 at 4:27 PM, Justin Bogner <mail at justinbogner.com>
wrote:
> Kostya Serebryany <kcc at google.com> writes:
> > On Tue, Aug 29, 2017 at 4:13 PM, Justin Bogner <mail at
justinbogner.com>
> > wrote:
> >
> >> Kostya Serebryany <kcc at google.com> writes:
> >> > Hi,
> >> >
> >> > We have several llvm fuzz targets running on OSS-Fuzz, a
continuous
> >> > automated fuzzing service:
> >> > https://github.com/google/oss-fuzz
> >> > https://www.usenix.org/sites/default/files/conference/
> protected-files/
> >> usenixsecurity17_slides_serebryany.pdf
> >> >
> >> > It has reported a few bugs in cxa_demangler, clang, and
dwarfdump
> >> already,
> >> > and we expect to add more fuzz targets to it soon
(llvm-isel-fuzzer,
> >> > clang-format-fuzzer, ...)
> >> >
> >> > A question to everyone: how do we report these bugs properly?
> >> > OSS-Fuzz files bugs automatically into a separate bug
tracker, it can
> not
> >> > file bugs to bugzilla.
> >> > By default, the bug reports are private for security reasons,
and only
> >> > those CC-ed explicitly can see them.
> >> >
> >> > Should we make the bug reports public by default?
> >> > We can set things differently for the llvm project (llvm,
clang, etc)
> >> and
> >> > libcxxabi (demangler):
> >> > https://github.com/google/oss-fuzz/tree/master/projects/llvm
> >> > https://github.com/google/oss-fuzz/tree/master/projects/
> llvm_libcxxabi
> >>
> >> At least some of these should probably just be public by default.
Things
> >> like llvm-isel-fuzzer or clang-fuzzer aren't really looking
for security
> >> bugs, so I wouldn't expect them to find stuff that falls under
the
> >> responsible disclosure umbrella.
> >>
> >
> > So, how about making all LLVM bugs public by default and leaving
> > cxa_demangler bugs private?
> > (I can't make it finer-grained, see below)
>
> This sounds good to me.
>
> >>
> >> This should be thought about on a case by case basis, of course.
> >>
> >> > Should we automatically CC the bugs to any of the llvm
maliing lists
> >> (e.g.
> >> > llvm-dev)?
> >>
> >> Perhaps we could CC them to llvm-bugs? That's the same list
that new
> >> bugzilla bugs are announced to.
> >>
> >
> > Ah, good idea.
> > Unless someone objects I'll add llvm-bugs to the spam^W list :)
> >
> >>
> >> > If a bug is CC-ed to a list, everyone will see the bug report
summary
> in
> >> > e-mail,
> >> > but if the bug remains private the reproducer for the bug
will remain
> >> > private.
> >> >
> >> > Who wants to be CC-ed explicitly?
> >> > (please add yourself to
> >> > https://github.com/google/oss-fuzz/blob/master/projects/
> >> llvm/project.yaml)
> >>
> >> Can this be set up to CC per-fuzz-target or so? I'm sure some
people are
> >> interested in, say, clang, but not necessarily cxa_demangler, or
> >> vice-versa.
> >>
> >
> > Sadly, no.
> >
> > We can distinguish llvm_cxxabi (cxa_demangler) from everything else
> because
> > these are currently two independent projects on oss-fuzz.
> > Making it finer-grained would require setting up separate oss-fuzz
> projects
> > which is harder to maintain and would not be welcome on oss-fuzz side.
> > The automatic e-mails announce the fuzz target's name, so filters
will be
> > easy to set up.
>
> Fair enough.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170830/c9dafff0/attachment-0001.html>