llvm dev - Sep 2015 - RFC: LTO should use -disable-llvm-verifier

On Thu, Sep 3, 2015 at 11:45 PM Mehdi Amini <mehdi.amini at apple.com>
wrote:
> Hi,
>
> > On Sep 2, 2015, at 7:31 PM, Peter Collingbourne via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > On Thu, Sep 03, 2015 at 01:10:42AM +0000, Eric Christopher wrote:
> >> On Tue, Sep 1, 2015 at 10:43 AM Duncan P. N. Exon Smith <
> >> dexonsmith at apple.com> wrote:
> >>
> >>>
> >>>> On 2015-Aug-31, at 18:09, Eric Christopher <echristo at
gmail.com>
> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Mon, Aug 31, 2015 at 5:50 PM Duncan P. N. Exon Smith
<
> >>> dexonsmith at apple.com> wrote:
> >>>>
> >>>>> On 2015-Aug-31, at 12:21, Eric Christopher
<echristo at gmail.com>
> wrote:
> >>>>> Yep. This is where I was going :)
> >>>>
> >>>> Glad I found consensus, but I want to double-check that
this makes
> >>>> sense to add to the driver.  I didn't quite think
through the
> >>>> implications myself.
> >>>>
> >>>> Since the driver doesn't know if there's any
bitcode, or if LTO is
> >>>> going to be invoked, it seems like I'll have to change
the noasserts
> >>>> driver to *always* pass the option to the linker just in
case we are
> >>>> doing LTO.  Is this reasonable?
> >>>>
> >>>> Also, I realized that passing `-mllvm
-disable-llvm-verifier` to ld64
> >>>> is redundant... so I'm thinking `-mllvm
-disable-verify`.  Make
> >>>> sense?
> >>>>
> >>>> *sigh* Reasons to hate the driver interface again...
> >>>>
> >>>> I guess this is ok. Could possibly add it to the existing
terrible
> >>> "enable this pass" interface on liblto as well.
> >>>
> >>> The linker doesn't know whether clang was built with
asserts, though.
> >>>
> >>> We could just make it implicit: move the decision to libLTO
itself.
> Given
> >>> that clang and libLTO.dylib are different executables anyway
-- and you
> >>> might be interposing an asserts libLTO.dylib to use with an
installed
> clang
> >>> -- maybe it's even better?
> >>>
> >>>
> >> *nod* We could do that. Seems better than the alternative.
> >
> > +1
> >
> >>>> I don't suppose ld64 could move to a model like
we're talking about
> with
> >>> lld that pcc is working on?
> >>>
> >>> What specifically?
> >>
> >>
> >> Ah, using the C++ interface to handle everything and not using
libLTO at
> >> all.
> >>
> >> He can speak more to this though.
> >
> > The C++ interface is much more convenient for a C++ program to use,
but
> > clients need to revlock themselves to LLVM in order to use it.
> >
> > In theory the same does not apply to libLTO, but we do end up changing
it
> > occasionally to add new APIs which ld64 promptly starts using, so it
> isn't
> > clear how much ld64 gains by relying on libLTO,
>
> Backward/Forward compatibility?
> Drop any version of clang/libLTO and still be able to use the system
> provided linker on any version of OS X?
> Sounds like a valuable feature to me, which is what I believe was (is?)
> sought by the C API in general (but that’s another story).
>
>
Static linking against the parts of llvm you care about? There's nothing in
the ld64 build system that means it can't do this :)

And you already don't have forward compatibility because of what Peter said
in the first place. :)

Anyhow, I don't actually expect this to change. Also the C API has to
change, but yes that's a different thread.

-eric

> —
> Mehdi
>
>
> > or whether the burden on
> > in-tree clients is worth it (there are certainly a number of internal
> APIs
> > that are more clumsy as a result of needing to support libLTO's
API; see
> > e.g. llvm::splitCodeGen's return value).
> >
> > Thanks,
> > --
> > Peter
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/e277df4e/attachment.html>

> On Sep 4, 2015, at 12:22 AM, Eric Christopher <echristo at gmail.com>
wrote:
> 
> 
> 
> On Thu, Sep 3, 2015 at 11:45 PM Mehdi Amini <mehdi.amini at apple.com
<mailto:mehdi.amini at apple.com>> wrote:
> Hi,
> 
> > On Sep 2, 2015, at 7:31 PM, Peter Collingbourne via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
> >
> > On Thu, Sep 03, 2015 at 01:10:42AM +0000, Eric Christopher wrote:
> >> On Tue, Sep 1, 2015 at 10:43 AM Duncan P. N. Exon Smith <
> >> dexonsmith at apple.com <mailto:dexonsmith at apple.com>>
wrote:
> >>
> >>>
> >>>> On 2015-Aug-31, at 18:09, Eric Christopher <echristo at
gmail.com <mailto:echristo at gmail.com>> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Mon, Aug 31, 2015 at 5:50 PM Duncan P. N. Exon Smith
<
> >>> dexonsmith at apple.com <mailto:dexonsmith at
apple.com>> wrote:
> >>>>
> >>>>> On 2015-Aug-31, at 12:21, Eric Christopher
<echristo at gmail.com <mailto:echristo at gmail.com>> wrote:
> >>>>> Yep. This is where I was going :)
> >>>>
> >>>> Glad I found consensus, but I want to double-check that
this makes
> >>>> sense to add to the driver.  I didn't quite think
through the
> >>>> implications myself.
> >>>>
> >>>> Since the driver doesn't know if there's any
bitcode, or if LTO is
> >>>> going to be invoked, it seems like I'll have to change
the noasserts
> >>>> driver to *always* pass the option to the linker just in
case we are
> >>>> doing LTO.  Is this reasonable?
> >>>>
> >>>> Also, I realized that passing `-mllvm
-disable-llvm-verifier` to ld64
> >>>> is redundant... so I'm thinking `-mllvm
-disable-verify`.  Make
> >>>> sense?
> >>>>
> >>>> *sigh* Reasons to hate the driver interface again...
> >>>>
> >>>> I guess this is ok. Could possibly add it to the existing
terrible
> >>> "enable this pass" interface on liblto as well.
> >>>
> >>> The linker doesn't know whether clang was built with
asserts, though.
> >>>
> >>> We could just make it implicit: move the decision to libLTO
itself.  Given
> >>> that clang and libLTO.dylib are different executables anyway
-- and you
> >>> might be interposing an asserts libLTO.dylib to use with an
installed clang
> >>> -- maybe it's even better?
> >>>
> >>>
> >> *nod* We could do that. Seems better than the alternative.
> >
> > +1
> >
> >>>> I don't suppose ld64 could move to a model like
we're talking about with
> >>> lld that pcc is working on?
> >>>
> >>> What specifically?
> >>
> >>
> >> Ah, using the C++ interface to handle everything and not using
libLTO at
> >> all.
> >>
> >> He can speak more to this though.
> >
> > The C++ interface is much more convenient for a C++ program to use,
but
> > clients need to revlock themselves to LLVM in order to use it.
> >
> > In theory the same does not apply to libLTO, but we do end up changing
it
> > occasionally to add new APIs which ld64 promptly starts using, so it
isn't
> > clear how much ld64 gains by relying on libLTO,
> 
> Backward/Forward compatibility?
> Drop any version of clang/libLTO and still be able to use the system
provided linker on any version of OS X?
> Sounds like a valuable feature to me, which is what I believe was (is?)
sought by the C API in general (but that’s another story).
> 
> 
> Static linking against the parts of llvm you care about? There's
nothing in the ld64 build system that means it can't do this :)
I’m not sure how is that supposed to work?
I drop a new clang/libLTO on the system, clang generates bitcode, and libLTO
handles it.

If ld64 was statically linked against LLVM, it couldn’t read the new bitcode
right?

— 
Mehdi


> 
> And you already don't have forward compatibility because of what Peter
said in the first place. :)
> 
> Anyhow, I don't actually expect this to change. Also the C API has to
change, but yes that's a different thread.
> 
> -eric
>  
> —
> Mehdi
> 
> 
> > or whether the burden on
> > in-tree clients is worth it (there are certainly a number of internal
APIs
> > that are more clumsy as a result of needing to support libLTO's
API; see
> > e.g. llvm::splitCodeGen's return value).
> >
> > Thanks,
> > --
> > Peter
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/c3ac5f7c/attachment.html>

On Fri, Sep 4, 2015 at 12:48 AM Mehdi Amini <mehdi.amini at apple.com>
wrote:
> On Sep 4, 2015, at 12:22 AM, Eric Christopher <echristo at gmail.com>
wrote:
>
>
>
> On Thu, Sep 3, 2015 at 11:45 PM Mehdi Amini <mehdi.amini at
apple.com> wrote:
>
>> Hi,
>>
>> > On Sep 2, 2015, at 7:31 PM, Peter Collingbourne via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>> >
>> > On Thu, Sep 03, 2015 at 01:10:42AM +0000, Eric Christopher wrote:
>> >> On Tue, Sep 1, 2015 at 10:43 AM Duncan P. N. Exon Smith <
>> >> dexonsmith at apple.com> wrote:
>> >>
>> >>>
>> >>>> On 2015-Aug-31, at 18:09, Eric Christopher
<echristo at gmail.com>
>> wrote:
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Mon, Aug 31, 2015 at 5:50 PM Duncan P. N. Exon
Smith <
>> >>> dexonsmith at apple.com> wrote:
>> >>>>
>> >>>>> On 2015-Aug-31, at 12:21, Eric Christopher
<echristo at gmail.com>
>> wrote:
>> >>>>> Yep. This is where I was going :)
>> >>>>
>> >>>> Glad I found consensus, but I want to double-check
that this makes
>> >>>> sense to add to the driver.  I didn't quite think
through the
>> >>>> implications myself.
>> >>>>
>> >>>> Since the driver doesn't know if there's any
bitcode, or if LTO is
>> >>>> going to be invoked, it seems like I'll have to
change the noasserts
>> >>>> driver to *always* pass the option to the linker just
in case we are
>> >>>> doing LTO.  Is this reasonable?
>> >>>>
>> >>>> Also, I realized that passing `-mllvm
-disable-llvm-verifier` to ld64
>> >>>> is redundant... so I'm thinking `-mllvm
-disable-verify`.  Make
>> >>>> sense?
>> >>>>
>> >>>> *sigh* Reasons to hate the driver interface again...
>> >>>>
>> >>>> I guess this is ok. Could possibly add it to the
existing terrible
>> >>> "enable this pass" interface on liblto as well.
>> >>>
>> >>> The linker doesn't know whether clang was built with
asserts, though.
>> >>>
>> >>> We could just make it implicit: move the decision to
libLTO itself.
>> Given
>> >>> that clang and libLTO.dylib are different executables
anyway -- and
>> you
>> >>> might be interposing an asserts libLTO.dylib to use with
an installed
>> clang
>> >>> -- maybe it's even better?
>> >>>
>> >>>
>> >> *nod* We could do that. Seems better than the alternative.
>> >
>> > +1
>> >
>> >>>> I don't suppose ld64 could move to a model like
we're talking about
>> with
>> >>> lld that pcc is working on?
>> >>>
>> >>> What specifically?
>> >>
>> >>
>> >> Ah, using the C++ interface to handle everything and not using
libLTO
>> at
>> >> all.
>> >>
>> >> He can speak more to this though.
>> >
>> > The C++ interface is much more convenient for a C++ program to
use, but
>> > clients need to revlock themselves to LLVM in order to use it.
>> >
>> > In theory the same does not apply to libLTO, but we do end up
changing
>> it
>> > occasionally to add new APIs which ld64 promptly starts using, so
it
>> isn't
>> > clear how much ld64 gains by relying on libLTO,
>>
>> Backward/Forward compatibility?
>> Drop any version of clang/libLTO and still be able to use the system
>> provided linker on any version of OS X?
>> Sounds like a valuable feature to me, which is what I believe was (is?)
>> sought by the C API in general (but that’s another story).
>>
>>
> Static linking against the parts of llvm you care about? There's
nothing
> in the ld64 build system that means it can't do this :)
>
>
> I’m not sure how is that supposed to work?
> I drop a new clang/libLTO on the system, clang generates bitcode, and
> libLTO handles it.
>
> If ld64 was statically linked against LLVM, it couldn’t read the new
> bitcode right?
>
>
Well sure, that's what Peter meant when he said revlocked. :)

-eric

> —
> Mehdi
>
>
>
>
> And you already don't have forward compatibility because of what Peter
> said in the first place. :)
>
>
> Anyhow, I don't actually expect this to change. Also the C API has to
> change, but yes that's a different thread.
>
> -eric
>
>
>> —
>> Mehdi
>>
>>
>> > or whether the burden on
>> > in-tree clients is worth it (there are certainly a number of
internal
>> APIs
>> > that are more clumsy as a result of needing to support
libLTO's API; see
>> > e.g. llvm::splitCodeGen's return value).
>> >
>> > Thanks,
>> > --
>> > Peter
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > llvm-dev at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/48d349a4/attachment.html>