llvm dev - May 2017 - moving libfuzzer to compiler-rt?

On Tue, May 2, 2017 at 9:34 PM, Justin Bogner <mail at justinbogner.com>
wrote:
> Reid Kleckner via llvm-dev <llvm-dev at lists.llvm.org> writes:
> > On Tue, May 2, 2017 at 4:28 PM, Kostya Serebryany via llvm-dev <
> > llvm-dev at lists.llvm.org> wrote:
> >
> >> On Tue, May 2, 2017 at 12:26 PM, George Karpenkov <ekarpenkov
at apple.com
> >
> >> wrote:
> >>
> >>> From my understanding, all these problems can be solved
entirely
> >>>
> >> by moving libfuzzer to compiler-rt, where (other) sanitizers
already
> >>> reside.
> >>>
> >>
> >> Yes, that might be a reasonable thing to do.
> >> I am trying to remember the reasons why we've put libFuzzer
into llvm
> and
> >> not into compiler-rt in the first place, and failing to do so.
> >>
> >
> > IIRC you thought libFuzzer would be more tightly coupled to LLVM
> > instrumentation, so the goal was to reap monorepo-like productivity
> > developments without waiting for it to happen?
> >
> > Maybe it was a licensing concern, UIUC vs dual UIUC/MIT?
> >
> >
> >> Any thoughts on the suggestion?
> >>>
> >>> It would be still possible to compile just libfuzzer with no
> >>> dependencies, by simply making a partial checkout from SVN,
> >>> and only the repo path would change.
> >>>
> >>
> >> This would cause some annoyance for me and maybe some users, but
nothing
> >> we can't tolerate.
> >> Still what are our other options?
> >>
> >> * wait for the mono repo to happen, then we'll be able to make
libFuzzer
> >> depend on clang. (Or no?)
> >> * move libFuzzer to a separate repo, parallel to compiler-rt? (not
a
> large
> >> win, just listing as a choice)
> >> * anything else?
> >>
> >> Does anyone see good reasons why libFuzzer should remain in llvm
repo
> (as
> >> opposed to moving it to compiler-rt)?
> >>
> >
> > I'd like to move libFuzzer out of llvm/lib/, since it's
definitely a
> > runtime library, and not compiler library infrastructure. compiler-rt
> seems
> > like the best place to put it for now, unless there are licensing
> concerns.
>
> Maybe we should just move it under llvm/runtimes and set it up with a
> fairly simple cmake config?

can we?
If we can, then we probably can also leave the code where it is, and tweak
the cmake to use just-built-clang. No?

> I think this makes it pretty trivial to
> build it with the just-built clang. It doesn't have terribly much to do
> with compiler-rt IMO - this isn't something that enables a compiler
> feature per se, it's more of its own thing.
>
> > Mechanically, we can do this in one history-preserving commit by
checking
> > out the entire SVN repo the way that the git-llvm script does. SVN
users
> > and users of https://github.com/llvm-project/llvm-project will still
see
> > the right history. The final monorepo will presumably also have
accurate
> > history.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170503/19740198/attachment.html>

Kostya Serebryany <kcc at google.com> writes:
> On Tue, May 2, 2017 at 9:34 PM, Justin Bogner <mail at
justinbogner.com> wrote:
>
>> Reid Kleckner via llvm-dev <llvm-dev at lists.llvm.org> writes:
>> > On Tue, May 2, 2017 at 4:28 PM, Kostya Serebryany via llvm-dev
<
>> > llvm-dev at lists.llvm.org> wrote:
>> >
>> >> On Tue, May 2, 2017 at 12:26 PM, George Karpenkov
<ekarpenkov at apple.com
>> >
>> >> wrote:
>> >>
>> >>> From my understanding, all these problems can be solved
entirely
>> >>>
>> >> by moving libfuzzer to compiler-rt, where (other) sanitizers
already
>> >>> reside.
>> >>>
>> >>
>> >> Yes, that might be a reasonable thing to do.
>> >> I am trying to remember the reasons why we've put
libFuzzer into llvm
>> and
>> >> not into compiler-rt in the first place, and failing to do so.
>> >>
>> >
>> > IIRC you thought libFuzzer would be more tightly coupled to LLVM
>> > instrumentation, so the goal was to reap monorepo-like
productivity
>> > developments without waiting for it to happen?
>> >
>> > Maybe it was a licensing concern, UIUC vs dual UIUC/MIT?
>> >
>> >
>> >> Any thoughts on the suggestion?
>> >>>
>> >>> It would be still possible to compile just libfuzzer with
no
>> >>> dependencies, by simply making a partial checkout from
SVN,
>> >>> and only the repo path would change.
>> >>>
>> >>
>> >> This would cause some annoyance for me and maybe some users,
but nothing
>> >> we can't tolerate.
>> >> Still what are our other options?
>> >>
>> >> * wait for the mono repo to happen, then we'll be able to
make libFuzzer
>> >> depend on clang. (Or no?)
>> >> * move libFuzzer to a separate repo, parallel to compiler-rt?
(not a
>> large
>> >> win, just listing as a choice)
>> >> * anything else?
>> >>
>> >> Does anyone see good reasons why libFuzzer should remain in
llvm repo
>> (as
>> >> opposed to moving it to compiler-rt)?
>> >>
>> >
>> > I'd like to move libFuzzer out of llvm/lib/, since it's
definitely a
>> > runtime library, and not compiler library infrastructure.
compiler-rt
>> seems
>> > like the best place to put it for now, unless there are licensing
>> concerns.
>>
>> Maybe we should just move it under llvm/runtimes and set it up with a
>> fairly simple cmake config?
>
> can we?
> If we can, then we probably can also leave the code where it is, and tweak
> the cmake to use just-built-clang. No?
Absolutely - this shouldn't be all that hard. I'd thought about taking a
crack at it a while ago but I haven't had the time.

The only complicated part is deciding what to do when we're not building
clang - should we build libFuzzer with the host compiler in this case,
or not build it at all?

>
> >
> > can we?
> > If we can, then we probably can also leave the code where it is, and
> tweak
> > the cmake to use just-built-clang. No?
>
> Absolutely - this shouldn't be all that hard. I'd thought about
taking a
> crack at it a while ago but I haven't had the time.
>
> The only complicated part is deciding what to do when we're not
building
> clang - should we build libFuzzer with the host compiler in this case,
> or not build it at all?
>
not build at all
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170503/cc2a0917/attachment.html>