llvm dev - Aug 2017 - llvm-mc-[dis]assemble-fuzzer status?

On Tue, Aug 22, 2017 at 4:34 PM, Kostya Serebryany <kcc at google.com>
wrote:
>
>
> On Tue, Aug 22, 2017 at 4:21 PM, George Karpenkov <ekarpenkov at
apple.com>
> wrote:
>
>> Hi,
>>
>> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
>> looking into updating the build code
>> for the libraries which use libFuzzer.
>>
>> I have tried to compile llvm-mc-assemble-fuzzer, and
>> llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
>> For the first one, the reason is that it refers to a nonexistent enum,
>> and for the second one I believe the reason is that it does not enclose
>> LLVMFuzzerTestOneInput in “extern ‘C’”.
>>
>> Are those libraries maintained and/or used?
>>
>> If yes, the code should be compilable, and ideally there should be a
>> buildbot.
>>
>
> "there should be a buildbot" is actually two different questions.
> 1. There should be a bot that builds the fuzz targets and runs them on a
> fixed set of inputs to ensure they don't bit-rot (and to use them as
> regression tests).
> This will require us to tweak the cmake machinery to allow building fuzz
> target with regular flags (no coverage).
> 2. There should also be a bot that actually runs continuous fuzzing.
> Our buildbots are not suitable for this, so I was planing to add the llvm
> fuzzers to OSS-Fuzz (https://github.com/google/oss-fuzz)
> We already run the cxa_demangler fuzzer there with quite a bit of success.
>

clang-fuzzer is now running on oss-fuzz, and here are two trophies so far:

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3134
ASSERT: ParmVarDeclBits.ScopeDepthOrObjCQuals == scopeDepth &&
"truncation!"
(haven't seen before)

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3133
llvm: ASSERT: DelayedTypos.empty() && "Uncorrected typos!"
An old friend: https://bugs.llvm.org/show_bug.cgi?id=21905

I'll add clang-proto-fuzzer soon.

Which other fuzz targets are worth adding to oss-fuzz?

Who else wants to be automatically CC-ed to all trophies?
(I'll need to add your e-mail here:
https://github.com/google/oss-fuzz/blob/master/projects/llvm/project.yaml)

>
> I hope Daniel can answer the other questions.
>
>
>> If no, maybe we should remove it, or move it to a separate repository.
>>
>> Thanks,
>> George
>>
>> (sorry for starting multiple threads, I believe this way it is more
>> convenient to keep track of tasks)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170823/0f3eccc8/attachment.html>

Kostya Serebryany <kcc at google.com> writes:
> On Tue, Aug 22, 2017 at 4:34 PM, Kostya Serebryany <kcc at
google.com> wrote:
>
>>
>>
>> On Tue, Aug 22, 2017 at 4:21 PM, George Karpenkov <ekarpenkov at
apple.com>
>> wrote:
>>
>>> Hi,
>>>
>>> As a part of a recent move of libFuzzer from LLVM to compiler-rt I
am
>>> looking into updating the build code
>>> for the libraries which use libFuzzer.
>>>
>>> I have tried to compile llvm-mc-assemble-fuzzer, and
>>> llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
>>> For the first one, the reason is that it refers to a nonexistent
enum,
>>> and for the second one I believe the reason is that it does not
enclose
>>> LLVMFuzzerTestOneInput in “extern ‘C’”.
>>>
>>> Are those libraries maintained and/or used?
>>>
>>> If yes, the code should be compilable, and ideally there should be
a
>>> buildbot.
>>>
>>
>> "there should be a buildbot" is actually two different
questions.
>> 1. There should be a bot that builds the fuzz targets and runs them on
a
>> fixed set of inputs to ensure they don't bit-rot (and to use them
as
>> regression tests).
>> This will require us to tweak the cmake machinery to allow building
fuzz
>> target with regular flags (no coverage).
>> 2. There should also be a bot that actually runs continuous fuzzing.
>> Our buildbots are not suitable for this, so I was planing to add the
llvm
>> fuzzers to OSS-Fuzz (https://github.com/google/oss-fuzz)
>> We already run the cxa_demangler fuzzer there with quite a bit of
success.
>
> clang-fuzzer is now running on oss-fuzz, and here are two trophies so far:
>
> https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3134
> ASSERT: ParmVarDeclBits.ScopeDepthOrObjCQuals == scopeDepth &&
"truncation!"
> (haven't seen before)
>
> https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3133
> llvm: ASSERT: DelayedTypos.empty() && "Uncorrected
typos!"
> An old friend: https://bugs.llvm.org/show_bug.cgi?id=21905
>
> I'll add clang-proto-fuzzer soon.
>
> Which other fuzz targets are worth adding to oss-fuzz?
I'd like llvm-isel-fuzzer to be added once its committed (which should
be as soon as LLVM fuzzers work in release builds again). One potential
issue is that llvm-isel-fuzzer is more of a collection of fuzzers, and
it needs some arguments to run (ie, to choose the backend).
> Who else wants to be automatically CC-ed to all trophies?
> (I'll need to add your e-mail here:
> https://github.com/google/oss-fuzz/blob/master/projects/llvm/project.yaml)
>
>
>>
>> I hope Daniel can answer the other questions.
>>
>>
>>> If no, maybe we should remove it, or move it to a separate
repository.
>>>
>>> Thanks,
>>> George
>>>
>>> (sorry for starting multiple threads, I believe this way it is more
>>> convenient to keep track of tasks)
>>
>>
>>

>
>
> I'd like llvm-isel-fuzzer to be added once its committed

consider it done (once it's there)

> (which should
> be as soon as LLVM fuzzers work in release builds again). One potential
> issue is that llvm-isel-fuzzer is more of a collection of fuzzers, and
> it needs some arguments to run (ie, to choose the backend).
>
I have the same problem with clang-proto-fuzzer, which uses the same
approach with flags as llvm-isel-fuzzer.

The solution I was thinking about is (drum roll!) to encode the flags in
the binary name, e.g.
"./llvm-isel-fuzzer,-flag1,-flag2" and then read these flags from
argv[0]
in LLVMFuzzerInitialize()

Then in oss-fuzz build.sh we will just do this:
for flags in -flag1a,-flag1b -flag2a,-flag2b; do
  cp llvm-isel-fuzzer $OUT/llvm-isel-fuzzer,$flags
done

>
> > Who else wants to be automatically CC-ed to all trophies?
> > (I'll need to add your e-mail here:
> > https://github.com/google/oss-fuzz/blob/master/projects/
> llvm/project.yaml)
> >
> >
> >>
> >> I hope Daniel can answer the other questions.
> >>
> >>
> >>> If no, maybe we should remove it, or move it to a separate
repository.
> >>>
> >>> Thanks,
> >>> George
> >>>
> >>> (sorry for starting multiple threads, I believe this way it is
more
> >>> convenient to keep track of tasks)
> >>
> >>
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170824/6e5d8379/attachment.html>