llvm dev - Aug 2017 - how to auto-report LLVM bugs found by fuzzing?

Hi,

We have several llvm fuzz targets running on OSS-Fuzz, a continuous
automated fuzzing service:
https://github.com/google/oss-fuzz
https://www.usenix.org/sites/default/files/conference/protected-files/usenixsecurity17_slides_serebryany.pdf

It has reported a few bugs in cxa_demangler, clang, and dwarfdump already,
and we expect to add more fuzz targets to it soon (llvm-isel-fuzzer,
clang-format-fuzzer, ...)

A question to everyone: how do we report these bugs properly?
OSS-Fuzz files bugs automatically into a separate bug tracker, it can not
file bugs to bugzilla.
By default, the bug reports are private for security reasons, and only
those CC-ed explicitly can see them.

Should we make the bug reports public by default?
We can set things differently for the llvm project (llvm, clang, etc)  and
libcxxabi (demangler):
https://github.com/google/oss-fuzz/tree/master/projects/llvm
https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi

Should we automatically CC the bugs to any of the llvm maliing lists (e.g.
llvm-dev)?
If a bug is CC-ed to a list, everyone will see the bug report summary in
e-mail,
but if the bug remains private the reproducer for the bug will remain
private.

Who wants to be CC-ed explicitly?
(please add yourself to
https://github.com/google/oss-fuzz/blob/master/projects/llvm/project.yaml)

Examples of bug reports follow.

Thanks!

--kcc


dwarfdump:

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3216&q=label%3AProj-llvm%20dwarfdump-fuzzer&colspec=ID%20Type%20Component%20Status%20Proj%20Reported%20Owner%20Summary

Crash Type: ASSERT
Crash Address:
Crash State:
  result <= UINT32_MAX
  llvm::object::WasmObjectFile::parseStartSection
  llvm::object::WasmObjectFile::parseSection

Crash Type: Heap-buffer-overflow READ 1
Crash Address: 0x60200000009a
Crash State:
  llvm::object::WasmObjectFile::parseCustomSection
  llvm::object::WasmObjectFile::parseSection
  llvm::object::WasmObjectFile::WasmObjectFile

Crash Type: Heap-buffer-overflow READ 1
Crash Address: 0x604000000776
Crash State:
  llvm::StringMapImpl::LookupBucketFor
  std::pair<llvm::StringMapIterator<unsigned int>, bool>
llvm::StringMap<unsigned
  llvm::DWARFContext::create

Crash Type: Heap-buffer-overflow READ 4
Crash Address: 0x60300000011c
Crash State:
  llvm::identify_magic
  llvm::object::ObjectFile::createObjectFile
  _start

clang-fuzzer:
https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=label%3AProj-llvm+clang-fuzzer&colspec=ID+Type+Component+Status+Proj+Reported+Owner+Summary&cells=ids
Crash Type: Stack-buffer-overflow READ 1
Crash Address: 0x7f79e7b71760
Crash State:
  clang::Lexer::SkipLineComment
  clang::Lexer::LexTokenInternal
  clang::Lexer::Lex

Crash Type: Direct-leak
Crash Address:
Crash State:
  clang::Parser::ParseParameterDeclarationClause
  clang::Parser::ParseFunctionDeclarator
  clang::Parser::ParseDirectDeclarator


Crash Type: Stack-overflow
Crash Address: 0x7ffc78d69f48
Crash State:
  clang::StmtVisitorBase<clang::make_const_ptr, IntExprEvaluator,
bool>::Visit
  Evaluate
  IntExprEvaluator::VisitBinaryOperator

Crash Type: ASSERT
Crash Address:
Crash State:
  !Prev.isAmbiguous() && "Cannot have an ambiguity in
previous-declaration
lookup"
  DiagnoseInvalidRedeclaration
  clang::Sema::ActOnFunctionDeclarator


cxa_demangler:

https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=label%3AProj-llvm_libcxxabi&colspec=ID+Type+Component+Status+Proj+Reported+Owner+Summary&cells=ids


Crash Type: Heap-buffer-overflow READ 8
Crash Address: 0x619000000078
Crash State:
  __cxxabiv1::parse_encoding
  __cxxabiv1::demangle
  __cxa_demangle
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170829/26654c21/attachment.html>

Kostya Serebryany <kcc at google.com> writes:
> Hi,
>
> We have several llvm fuzz targets running on OSS-Fuzz, a continuous
> automated fuzzing service:
> https://github.com/google/oss-fuzz
>
https://www.usenix.org/sites/default/files/conference/protected-files/usenixsecurity17_slides_serebryany.pdf
>
> It has reported a few bugs in cxa_demangler, clang, and dwarfdump already,
> and we expect to add more fuzz targets to it soon (llvm-isel-fuzzer,
> clang-format-fuzzer, ...)
>
> A question to everyone: how do we report these bugs properly?
> OSS-Fuzz files bugs automatically into a separate bug tracker, it can not
> file bugs to bugzilla.
> By default, the bug reports are private for security reasons, and only
> those CC-ed explicitly can see them.
>
> Should we make the bug reports public by default?
> We can set things differently for the llvm project (llvm, clang, etc)  and
> libcxxabi (demangler):
> https://github.com/google/oss-fuzz/tree/master/projects/llvm
> https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi
At least some of these should probably just be public by default. Things
like llvm-isel-fuzzer or clang-fuzzer aren't really looking for security
bugs, so I wouldn't expect them to find stuff that falls under the
responsible disclosure umbrella.

This should be thought about on a case by case basis, of course.
> Should we automatically CC the bugs to any of the llvm maliing lists (e.g.
> llvm-dev)?
Perhaps we could CC them to llvm-bugs? That's the same list that new
bugzilla bugs are announced to.
> If a bug is CC-ed to a list, everyone will see the bug report summary in
> e-mail,
> but if the bug remains private the reproducer for the bug will remain
> private.
>
> Who wants to be CC-ed explicitly?
> (please add yourself to
> https://github.com/google/oss-fuzz/blob/master/projects/llvm/project.yaml)
Can this be set up to CC per-fuzz-target or so? I'm sure some people are
interested in, say, clang, but not necessarily cxa_demangler, or
vice-versa.
> Examples of bug reports follow.
>
> Thanks!
>
> --kcc
>
>
> dwarfdump:
>
>
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3216&q=label%3AProj-llvm%20dwarfdump-fuzzer&colspec=ID%20Type%20Component%20Status%20Proj%20Reported%20Owner%20Summary
>
> Crash Type: ASSERT
> Crash Address:
> Crash State:
>   result <= UINT32_MAX
>   llvm::object::WasmObjectFile::parseStartSection
>   llvm::object::WasmObjectFile::parseSection
>
> Crash Type: Heap-buffer-overflow READ 1
> Crash Address: 0x60200000009a
> Crash State:
>   llvm::object::WasmObjectFile::parseCustomSection
>   llvm::object::WasmObjectFile::parseSection
>   llvm::object::WasmObjectFile::WasmObjectFile
>
> Crash Type: Heap-buffer-overflow READ 1
> Crash Address: 0x604000000776
> Crash State:
>   llvm::StringMapImpl::LookupBucketFor
>   std::pair<llvm::StringMapIterator<unsigned int>, bool>
> llvm::StringMap<unsigned
>   llvm::DWARFContext::create
>
> Crash Type: Heap-buffer-overflow READ 4
> Crash Address: 0x60300000011c
> Crash State:
>   llvm::identify_magic
>   llvm::object::ObjectFile::createObjectFile
>   _start
>
> clang-fuzzer:
>
https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=label%3AProj-llvm+clang-fuzzer&colspec=ID+Type+Component+Status+Proj+Reported+Owner+Summary&cells=ids
> Crash Type: Stack-buffer-overflow READ 1
> Crash Address: 0x7f79e7b71760
> Crash State:
>   clang::Lexer::SkipLineComment
>   clang::Lexer::LexTokenInternal
>   clang::Lexer::Lex
>
> Crash Type: Direct-leak
> Crash Address:
> Crash State:
>   clang::Parser::ParseParameterDeclarationClause
>   clang::Parser::ParseFunctionDeclarator
>   clang::Parser::ParseDirectDeclarator
>
>
> Crash Type: Stack-overflow
> Crash Address: 0x7ffc78d69f48
> Crash State:
>   clang::StmtVisitorBase<clang::make_const_ptr, IntExprEvaluator,
> bool>::Visit
>   Evaluate
>   IntExprEvaluator::VisitBinaryOperator
>
> Crash Type: ASSERT
> Crash Address:
> Crash State:
>   !Prev.isAmbiguous() && "Cannot have an ambiguity in
previous-declaration
> lookup"
>   DiagnoseInvalidRedeclaration
>   clang::Sema::ActOnFunctionDeclarator
>
>
> cxa_demangler:
>
>
https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=label%3AProj-llvm_libcxxabi&colspec=ID+Type+Component+Status+Proj+Reported+Owner+Summary&cells=ids
>
>
> Crash Type: Heap-buffer-overflow READ 8
> Crash Address: 0x619000000078
> Crash State:
>   __cxxabiv1::parse_encoding
>   __cxxabiv1::demangle
>   __cxa_demangle

On Tue, Aug 29, 2017 at 4:13 PM, Justin Bogner <mail at justinbogner.com>
wrote:
> Kostya Serebryany <kcc at google.com> writes:
> > Hi,
> >
> > We have several llvm fuzz targets running on OSS-Fuzz, a continuous
> > automated fuzzing service:
> > https://github.com/google/oss-fuzz
> > https://www.usenix.org/sites/default/files/conference/protected-files/
> usenixsecurity17_slides_serebryany.pdf
> >
> > It has reported a few bugs in cxa_demangler, clang, and dwarfdump
> already,
> > and we expect to add more fuzz targets to it soon (llvm-isel-fuzzer,
> > clang-format-fuzzer, ...)
> >
> > A question to everyone: how do we report these bugs properly?
> > OSS-Fuzz files bugs automatically into a separate bug tracker, it can
not
> > file bugs to bugzilla.
> > By default, the bug reports are private for security reasons, and only
> > those CC-ed explicitly can see them.
> >
> > Should we make the bug reports public by default?
> > We can set things differently for the llvm project (llvm, clang, etc)
> and
> > libcxxabi (demangler):
> > https://github.com/google/oss-fuzz/tree/master/projects/llvm
> > https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi
>
> At least some of these should probably just be public by default. Things
> like llvm-isel-fuzzer or clang-fuzzer aren't really looking for
security
> bugs, so I wouldn't expect them to find stuff that falls under the
> responsible disclosure umbrella.
>
So, how about making all LLVM bugs public by default and leaving
cxa_demangler bugs private?
(I can't make it finer-grained, see below)

>
> This should be thought about on a case by case basis, of course.
>
> > Should we automatically CC the bugs to any of the llvm maliing lists
> (e.g.
> > llvm-dev)?
>
> Perhaps we could CC them to llvm-bugs? That's the same list that new
> bugzilla bugs are announced to.
>
Ah, good idea.
Unless someone objects I'll add llvm-bugs to the spam^W list :)

>
> > If a bug is CC-ed to a list, everyone will see the bug report summary
in
> > e-mail,
> > but if the bug remains private the reproducer for the bug will remain
> > private.
> >
> > Who wants to be CC-ed explicitly?
> > (please add yourself to
> > https://github.com/google/oss-fuzz/blob/master/projects/
> llvm/project.yaml)
>
> Can this be set up to CC per-fuzz-target or so? I'm sure some people
are
> interested in, say, clang, but not necessarily cxa_demangler, or
> vice-versa.
>
Sadly, no.

We can distinguish llvm_cxxabi (cxa_demangler) from everything else because
these are currently two independent projects on oss-fuzz.
Making it finer-grained would require setting up separate oss-fuzz projects
which is harder to maintain and would not be welcome on oss-fuzz side.
The automatic e-mails announce the fuzz target's name, so filters will be
easy to set up.

>
> > Examples of bug reports follow.
> >
> > Thanks!
> >
> > --kcc
> >
> >
> > dwarfdump:
> >
> > https://bugs.chromium.org/p/oss-fuzz/issues/detail?id>
3216&q=label%3AProj-llvm%20dwarfdump-fuzzer&colspec=ID%
> 20Type%20Component%20Status%20Proj%20Reported%20Owner%20Summary
> >
> > Crash Type: ASSERT
> > Crash Address:
> > Crash State:
> >   result <= UINT32_MAX
> >   llvm::object::WasmObjectFile::parseStartSection
> >   llvm::object::WasmObjectFile::parseSection
> >
> > Crash Type: Heap-buffer-overflow READ 1
> > Crash Address: 0x60200000009a
> > Crash State:
> >   llvm::object::WasmObjectFile::parseCustomSection
> >   llvm::object::WasmObjectFile::parseSection
> >   llvm::object::WasmObjectFile::WasmObjectFile
> >
> > Crash Type: Heap-buffer-overflow READ 1
> > Crash Address: 0x604000000776
> > Crash State:
> >   llvm::StringMapImpl::LookupBucketFor
> >   std::pair<llvm::StringMapIterator<unsigned int>, bool>
> > llvm::StringMap<unsigned
> >   llvm::DWARFContext::create
> >
> > Crash Type: Heap-buffer-overflow READ 4
> > Crash Address: 0x60300000011c
> > Crash State:
> >   llvm::identify_magic
> >   llvm::object::ObjectFile::createObjectFile
> >   _start
> >
> > clang-fuzzer:
> > https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q>
label%3AProj-llvm+clang-fuzzer&colspec=ID+Type+Component+Status+Proj+
> Reported+Owner+Summary&cells=ids
> > Crash Type: Stack-buffer-overflow READ 1
> > Crash Address: 0x7f79e7b71760
> > Crash State:
> >   clang::Lexer::SkipLineComment
> >   clang::Lexer::LexTokenInternal
> >   clang::Lexer::Lex
> >
> > Crash Type: Direct-leak
> > Crash Address:
> > Crash State:
> >   clang::Parser::ParseParameterDeclarationClause
> >   clang::Parser::ParseFunctionDeclarator
> >   clang::Parser::ParseDirectDeclarator
> >
> >
> > Crash Type: Stack-overflow
> > Crash Address: 0x7ffc78d69f48
> > Crash State:
> >   clang::StmtVisitorBase<clang::make_const_ptr, IntExprEvaluator,
> > bool>::Visit
> >   Evaluate
> >   IntExprEvaluator::VisitBinaryOperator
> >
> > Crash Type: ASSERT
> > Crash Address:
> > Crash State:
> >   !Prev.isAmbiguous() && "Cannot have an ambiguity in
> previous-declaration
> > lookup"
> >   DiagnoseInvalidRedeclaration
> >   clang::Sema::ActOnFunctionDeclarator
> >
> >
> > cxa_demangler:
> >
> > https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q>
label%3AProj-llvm_libcxxabi&colspec=ID+Type+Component+
> Status+Proj+Reported+Owner+Summary&cells=ids
> >
> >
> > Crash Type: Heap-buffer-overflow READ 8
> > Crash Address: 0x619000000078
> > Crash State:
> >   __cxxabiv1::parse_encoding
> >   __cxxabiv1::demangle
> >   __cxa_demangle
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170829/3f4305aa/attachment.html>

On Tue, Aug 29, 2017 at 4:13 PM, Justin Bogner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> > Should we automatically CC the bugs to any of the llvm maliing lists
> (e.g.
> > llvm-dev)?
>
> Perhaps we could CC them to llvm-bugs? That's the same list that new
> bugzilla bugs are announced to.
>
Please don't, llvm-bugs is relatively low-traffic today. There will be a
mountain of fuzzer bugs. Right now llvm-bugs is only issues that actual
humans took the time to file. I volunteer my time to help humans with real
issues, but I don't necessarily want to triage every fuzzer bug ever.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170829/875b69a7/attachment.html>