llvm dev - Aug 2017 - llvm-mc-[dis]assemble-fuzzer status?

Hi,

As a part of a recent move of libFuzzer from LLVM to compiler-rt I am looking
into updating the build code
for the libraries which use libFuzzer.

I have tried to compile llvm-mc-assemble-fuzzer, and llvm-mc-disassemble-fuzzer,
and I couldn’t build either of those.
For the first one, the reason is that it refers to a nonexistent enum,
and for the second one I believe the reason is that it does not enclose
LLVMFuzzerTestOneInput in “extern ‘C’”.

Are those libraries maintained and/or used?

If yes, the code should be compilable, and ideally there should be a buildbot.
If no, maybe we should remove it, or move it to a separate repository.

Thanks,
George

(sorry for starting multiple threads, I believe this way it is more convenient
to keep track of tasks)

George Karpenkov <ekarpenkov at apple.com> writes:
> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
> looking into updating the build code
> for the libraries which use libFuzzer.
>
> I have tried to compile llvm-mc-assemble-fuzzer, and
> llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
> For the first one, the reason is that it refers to a nonexistent enum,
> and for the second one I believe the reason is that it does not
> enclose LLVMFuzzerTestOneInput in “extern ‘C’”.
>
> Are those libraries maintained and/or used?
I believe both of these worked a couple of months back when I last tried
them.
> If yes, the code should be compilable, and ideally there should be a
buildbot.
> If no, maybe we should remove it, or move it to a separate repository.
Now that libFuzzer is part of the clang toolchain it should be much
easier to get bots up that are building these tools. Previously it was a
bit awkward.

I think it makes sense to fix these ones.
> Thanks,
> George
>
> (sorry for starting multiple threads, I believe this way it is more
> convenient to keep track of tasks)

On Tue, Aug 22, 2017 at 4:21 PM, George Karpenkov <ekarpenkov at
apple.com>
wrote:
> Hi,
>
> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
> looking into updating the build code
> for the libraries which use libFuzzer.
>
> I have tried to compile llvm-mc-assemble-fuzzer, and
> llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
> For the first one, the reason is that it refers to a nonexistent enum,
> and for the second one I believe the reason is that it does not enclose
> LLVMFuzzerTestOneInput in “extern ‘C’”.
>
> Are those libraries maintained and/or used?
>
> If yes, the code should be compilable, and ideally there should be a
> buildbot.
>
"there should be a buildbot" is actually two different questions.
1. There should be a bot that builds the fuzz targets and runs them on a
fixed set of inputs to ensure they don't bit-rot (and to use them as
regression tests).
This will require us to tweak the cmake machinery to allow building fuzz
target with regular flags (no coverage).
2. There should also be a bot that actually runs continuous fuzzing.
Our buildbots are not suitable for this, so I was planing to add the llvm
fuzzers to OSS-Fuzz (https://github.com/google/oss-fuzz)
We already run the cxa_demangler fuzzer there with quite a bit of success.

I hope Daniel can answer the other questions.

> If no, maybe we should remove it, or move it to a separate repository.
>
> Thanks,
> George
>
> (sorry for starting multiple threads, I believe this way it is more
> convenient to keep track of tasks)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170822/7556b8e8/attachment.html>

George,

Thanks for doing the work to move libFuzzer to compiler-rt.

I probably touched these more recently than most and either didn't deliver
a complete patch or it's rotted since then.  In any case I haven't
gotten a
chance to leverage it.  But I'd like the idea of it arriving intact in the
move to compiler-rt.  If I'm able to get it back and running by the end of
the week, would that be adequate?

Regarding a buildbot -- I think that makes sense.  I naively assumed that
it would be a part of the default build set for the "all" target when
building llvm.  I figured that if there were ever any regression, that I or
some other owner would be notified.

On Tue, Aug 22, 2017 at 6:21 PM, George Karpenkov via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
> looking into updating the build code
> for the libraries which use libFuzzer.
>
> I have tried to compile llvm-mc-assemble-fuzzer, and
> llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
> For the first one, the reason is that it refers to a nonexistent enum,
> and for the second one I believe the reason is that it does not enclose
> LLVMFuzzerTestOneInput in “extern ‘C’”.
>
> Are those libraries maintained and/or used?
>
> If yes, the code should be compilable, and ideally there should be a
> buildbot.
> If no, maybe we should remove it, or move it to a separate repository.
>
> Thanks,
> George
>
> (sorry for starting multiple threads, I believe this way it is more
> convenient to keep track of tasks)
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>


-- 
-Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170822/4a8aab5e/attachment.html>

> On Aug 22, 2017, at 4:35 PM, Brian Cain <brian.cain at gmail.com>
wrote:
> 
> George,
> 
> Thanks for doing the work to move libFuzzer to compiler-rt.
> 
> I probably touched these more recently than most and either didn't
deliver a complete patch or it's rotted since then.  In any case I
haven't gotten a chance to leverage it.  But I'd like the idea of it
arriving intact in the move to compiler-rt.  If I'm able to get it back and
running by the end of the week, would that be adequate?
> 
> Regarding a buildbot -- I think that makes sense.  I naively assumed that
it would be a part of the default build set for the "all" target when
building llvm.  I figured that if there were ever any regression, that I or some
other owner would be notified.
Hi Brian,

Great, thanks!
I’m not sure why it’s not build: maybe because we never run “all” when
LLVM_USE_SANITIZE_COVERAGE is set, which is required that to build that library.
> 
> On Tue, Aug 22, 2017 at 6:21 PM, George Karpenkov via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hi,
> 
> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
looking into updating the build code
> for the libraries which use libFuzzer.
> 
> I have tried to compile llvm-mc-assemble-fuzzer, and
llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
> For the first one, the reason is that it refers to a nonexistent enum,
> and for the second one I believe the reason is that it does not enclose
LLVMFuzzerTestOneInput in “extern ‘C’”.
> 
> Are those libraries maintained and/or used?
> 
> If yes, the code should be compilable, and ideally there should be a
buildbot.
> If no, maybe we should remove it, or move it to a separate repository.
> 
> Thanks,
> George
> 
> (sorry for starting multiple threads, I believe this way it is more
convenient to keep track of tasks)
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> 
> -- 
> -Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170822/212fe1d8/attachment.html>

> On Aug 22, 2017, at 4:34 PM, Kostya Serebryany <kcc at google.com>
wrote:
> 
> 
> 
> On Tue, Aug 22, 2017 at 4:21 PM, George Karpenkov <ekarpenkov at
apple.com <mailto:ekarpenkov at apple.com>> wrote:
> Hi,
> 
> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
looking into updating the build code
> for the libraries which use libFuzzer.
> 
> I have tried to compile llvm-mc-assemble-fuzzer, and
llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
> For the first one, the reason is that it refers to a nonexistent enum,
> and for the second one I believe the reason is that it does not enclose
LLVMFuzzerTestOneInput in “extern ‘C’”.
> 
> Are those libraries maintained and/or used?
> 
> If yes, the code should be compilable, and ideally there should be a
buildbot.
> 
> "there should be a buildbot" is actually two different questions.
> 1. There should be a bot that builds the fuzz targets and runs them on a
fixed set of inputs to ensure they don't bit-rot (and to use them as
regression tests).
I’ve just meant building them, not even necessarily running.
Then authors / people who make changes would notice, and it would get compiled.
> This will require us to tweak the cmake machinery to allow building fuzz
target with regular flags (no coverage).
I’m not sure why that would be necessary? We can have a checkout setup with
LLVM_USE_SANITIZERS=ON.
> 2. There should also be a bot that actually runs continuous fuzzing. 
> Our buildbots are not suitable for this, so I was planing to add the llvm
fuzzers to OSS-Fuzz (https://github.com/google/oss-fuzz
<https://github.com/google/oss-fuzz>)
> We already run the cxa_demangler fuzzer there with quite a bit of success. 
Right, that would be great as well!
> 
> I hope Daniel can answer the other questions. 
>  
> If no, maybe we should remove it, or move it to a separate repository.
> 
> Thanks,
> George
> 
> (sorry for starting multiple threads, I believe this way it is more
convenient to keep track of tasks)
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170822/6314d8a8/attachment.html>

On Tue, Aug 22, 2017 at 4:34 PM, Kostya Serebryany <kcc at google.com>
wrote:
>
>
> On Tue, Aug 22, 2017 at 4:21 PM, George Karpenkov <ekarpenkov at
apple.com>
> wrote:
>
>> Hi,
>>
>> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
>> looking into updating the build code
>> for the libraries which use libFuzzer.
>>
>> I have tried to compile llvm-mc-assemble-fuzzer, and
>> llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
>> For the first one, the reason is that it refers to a nonexistent enum,
>> and for the second one I believe the reason is that it does not enclose
>> LLVMFuzzerTestOneInput in “extern ‘C’”.
>>
>> Are those libraries maintained and/or used?
>>
>> If yes, the code should be compilable, and ideally there should be a
>> buildbot.
>>
>
> "there should be a buildbot" is actually two different questions.
> 1. There should be a bot that builds the fuzz targets and runs them on a
> fixed set of inputs to ensure they don't bit-rot (and to use them as
> regression tests).
> This will require us to tweak the cmake machinery to allow building fuzz
> target with regular flags (no coverage).
> 2. There should also be a bot that actually runs continuous fuzzing.
> Our buildbots are not suitable for this, so I was planing to add the llvm
> fuzzers to OSS-Fuzz (https://github.com/google/oss-fuzz)
> We already run the cxa_demangler fuzzer there with quite a bit of success.
>

clang-fuzzer is now running on oss-fuzz, and here are two trophies so far:

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3134
ASSERT: ParmVarDeclBits.ScopeDepthOrObjCQuals == scopeDepth &&
"truncation!"
(haven't seen before)

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3133
llvm: ASSERT: DelayedTypos.empty() && "Uncorrected typos!"
An old friend: https://bugs.llvm.org/show_bug.cgi?id=21905

I'll add clang-proto-fuzzer soon.

Which other fuzz targets are worth adding to oss-fuzz?

Who else wants to be automatically CC-ed to all trophies?
(I'll need to add your e-mail here:
https://github.com/google/oss-fuzz/blob/master/projects/llvm/project.yaml)

>
> I hope Daniel can answer the other questions.
>
>
>> If no, maybe we should remove it, or move it to a separate repository.
>>
>> Thanks,
>> George
>>
>> (sorry for starting multiple threads, I believe this way it is more
>> convenient to keep track of tasks)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170823/0f3eccc8/attachment.html>

(removed my @imgtec.com address since it no longer exists)

Sorry for the slow reply, it's a busy time for me right now.
> On 23 Aug 2017, at 00:21, George Karpenkov via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi,
> 
> As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
looking into updating the build code
> for the libraries which use libFuzzer.
> 
> I have tried to compile llvm-mc-assemble-fuzzer, and
llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
> For the first one, the reason is that it refers to a nonexistent enum,
I don't seem to be able to build this with cmake+ninja yet (I'm having
trouble recursing the compiler on macOS) but after manually building it... It
seems this broke at the start of August when the CodeModel argument was removed
from InitMCObjectFileInfo(). After removing that argument and adding an
'extern "C"' it at least compiles. I haven't had chance to
try running it yet.
> and for the second one I believe the reason is that it does not enclose
LLVMFuzzerTestOneInput in “extern ‘C’”.
I agree we need an "extern C" here. I'm not sure what changed to
make it required though.
> Are those libraries maintained and/or used?
I haven't used it for quite a while now. My original motivator was the Mips
assembler/disassembler being very buggy. I was using it to find crashes and
generate interesting test cases for round-trip testing of the
assembler/disassembler. Since then, the Mips MC layer has become much more
stable and I've also changed jobs.

That said, I'd like to set up a bot to make use of these tools, it's
mostly a matter of finding time for it. That's normally difficult but I
should be able to do that in the next few weeks.
> If yes, the code should be compilable, and ideally there should be a
buildbot.
> If no, maybe we should remove it, or move it to a separate repository.
> 
> Thanks,
> George
> 
> (sorry for starting multiple threads, I believe this way it is more
convenient to keep track of tasks)
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Fri, Aug 25, 2017 at 8:51 AM, Daniel Sanders <daniel_l_sanders at
apple.com>
wrote:
> (removed my @imgtec.com address since it no longer exists)
>
> Sorry for the slow reply, it's a busy time for me right now.
>
> > On 23 Aug 2017, at 00:21, George Karpenkov via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hi,
> >
> > As a part of a recent move of libFuzzer from LLVM to compiler-rt I am
> looking into updating the build code
> > for the libraries which use libFuzzer.
> >
> > I have tried to compile llvm-mc-assemble-fuzzer, and
> llvm-mc-disassemble-fuzzer, and I couldn’t build either of those.
> > For the first one, the reason is that it refers to a nonexistent enum,
>
> I don't seem to be able to build this with cmake+ninja yet (I'm
having
> trouble recursing the compiler on macOS) but after manually building it...
> It seems this broke at the start of August when the CodeModel argument was
> removed from InitMCObjectFileInfo(). After removing that argument and
> adding an 'extern "C"' it at least compiles. I
haven't had chance to try
> running it yet.
>
> > and for the second one I believe the reason is that it does not
enclose
> LLVMFuzzerTestOneInput in “extern ‘C’”.
>
> I agree we need an "extern C" here. I'm not sure what changed
to make it
> required though.
>
> > Are those libraries maintained and/or used?
>
> I haven't used it for quite a while now. My original motivator was the
> Mips assembler/disassembler being very buggy. I was using it to find
> crashes and generate interesting test cases for round-trip testing of the
> assembler/disassembler. Since then, the Mips MC layer has become much more
> stable and I've also changed jobs.
>
> That said, I'd like to set up a bot to make use of these tools,

As soon as these fuzz targets build, don't immediately crash, and have
someone who cares about them,
I can add them to OSS-Fuzz for automated continuous fuzzing.


> it's mostly a matter of finding time for it. That's normally
difficult but
> I should be able to do that in the next few weeks.
>
> > If yes, the code should be compilable, and ideally there should be a
> buildbot.
> > If no, maybe we should remove it, or move it to a separate repository.
> >
> > Thanks,
> > George
> >
> > (sorry for starting multiple threads, I believe this way it is more
> convenient to keep track of tasks)
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170825/6ea54ec5/attachment.html>