Eric Christopher via llvm-dev
2015-Sep-04 18:03 UTC
[llvm-dev] RFC: LTO should use -disable-llvm-verifier
On Fri, Sep 4, 2015 at 12:48 AM Mehdi Amini <mehdi.amini at apple.com> wrote:> On Sep 4, 2015, at 12:22 AM, Eric Christopher <echristo at gmail.com> wrote: > > > > On Thu, Sep 3, 2015 at 11:45 PM Mehdi Amini <mehdi.amini at apple.com> wrote: > >> Hi, >> >> > On Sep 2, 2015, at 7:31 PM, Peter Collingbourne via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> > >> > On Thu, Sep 03, 2015 at 01:10:42AM +0000, Eric Christopher wrote: >> >> On Tue, Sep 1, 2015 at 10:43 AM Duncan P. N. Exon Smith < >> >> dexonsmith at apple.com> wrote: >> >> >> >>> >> >>>> On 2015-Aug-31, at 18:09, Eric Christopher <echristo at gmail.com> >> wrote: >> >>>> >> >>>> >> >>>> >> >>>> On Mon, Aug 31, 2015 at 5:50 PM Duncan P. N. Exon Smith < >> >>> dexonsmith at apple.com> wrote: >> >>>> >> >>>>> On 2015-Aug-31, at 12:21, Eric Christopher <echristo at gmail.com> >> wrote: >> >>>>> Yep. This is where I was going :) >> >>>> >> >>>> Glad I found consensus, but I want to double-check that this makes >> >>>> sense to add to the driver. I didn't quite think through the >> >>>> implications myself. >> >>>> >> >>>> Since the driver doesn't know if there's any bitcode, or if LTO is >> >>>> going to be invoked, it seems like I'll have to change the noasserts >> >>>> driver to *always* pass the option to the linker just in case we are >> >>>> doing LTO. Is this reasonable? >> >>>> >> >>>> Also, I realized that passing `-mllvm -disable-llvm-verifier` to ld64 >> >>>> is redundant... so I'm thinking `-mllvm -disable-verify`. Make >> >>>> sense? >> >>>> >> >>>> *sigh* Reasons to hate the driver interface again... >> >>>> >> >>>> I guess this is ok. Could possibly add it to the existing terrible >> >>> "enable this pass" interface on liblto as well. >> >>> >> >>> The linker doesn't know whether clang was built with asserts, though. >> >>> >> >>> We could just make it implicit: move the decision to libLTO itself. >> Given >> >>> that clang and libLTO.dylib are different executables anyway -- and >> you >> >>> might be interposing an asserts libLTO.dylib to use with an installed >> clang >> >>> -- maybe it's even better? >> >>> >> >>> >> >> *nod* We could do that. Seems better than the alternative. >> > >> > +1 >> > >> >>>> I don't suppose ld64 could move to a model like we're talking about >> with >> >>> lld that pcc is working on? >> >>> >> >>> What specifically? >> >> >> >> >> >> Ah, using the C++ interface to handle everything and not using libLTO >> at >> >> all. >> >> >> >> He can speak more to this though. >> > >> > The C++ interface is much more convenient for a C++ program to use, but >> > clients need to revlock themselves to LLVM in order to use it. >> > >> > In theory the same does not apply to libLTO, but we do end up changing >> it >> > occasionally to add new APIs which ld64 promptly starts using, so it >> isn't >> > clear how much ld64 gains by relying on libLTO, >> >> Backward/Forward compatibility? >> Drop any version of clang/libLTO and still be able to use the system >> provided linker on any version of OS X? >> Sounds like a valuable feature to me, which is what I believe was (is?) >> sought by the C API in general (but that’s another story). >> >> > Static linking against the parts of llvm you care about? There's nothing > in the ld64 build system that means it can't do this :) > > > I’m not sure how is that supposed to work? > I drop a new clang/libLTO on the system, clang generates bitcode, and > libLTO handles it. > > If ld64 was statically linked against LLVM, it couldn’t read the new > bitcode right? > >Well sure, that's what Peter meant when he said revlocked. :) -eric> — > Mehdi > > > > > And you already don't have forward compatibility because of what Peter > said in the first place. :) > > > Anyhow, I don't actually expect this to change. Also the C API has to > change, but yes that's a different thread. > > -eric > > >> — >> Mehdi >> >> >> > or whether the burden on >> > in-tree clients is worth it (there are certainly a number of internal >> APIs >> > that are more clumsy as a result of needing to support libLTO's API; see >> > e.g. llvm::splitCodeGen's return value). >> > >> > Thanks, >> > -- >> > Peter >> > _______________________________________________ >> > LLVM Developers mailing list >> > llvm-dev at lists.llvm.org >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/48d349a4/attachment.html>
Mehdi Amini via llvm-dev
2015-Sep-04 18:13 UTC
[llvm-dev] RFC: LTO should use -disable-llvm-verifier
> On Sep 4, 2015, at 11:03 AM, Eric Christopher <echristo at gmail.com> wrote: > > > > On Fri, Sep 4, 2015 at 12:48 AM Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: >> On Sep 4, 2015, at 12:22 AM, Eric Christopher <echristo at gmail.com <mailto:echristo at gmail.com>> wrote: >> >> >> >> On Thu, Sep 3, 2015 at 11:45 PM Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: >> Hi, >> >> > On Sep 2, 2015, at 7:31 PM, Peter Collingbourne via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >> > >> > On Thu, Sep 03, 2015 at 01:10:42AM +0000, Eric Christopher wrote: >> >> On Tue, Sep 1, 2015 at 10:43 AM Duncan P. N. Exon Smith < >> >> dexonsmith at apple.com <mailto:dexonsmith at apple.com>> wrote: >> >> >> >>> >> >>>> On 2015-Aug-31, at 18:09, Eric Christopher <echristo at gmail.com <mailto:echristo at gmail.com>> wrote: >> >>>> >> >>>> >> >>>> >> >>>> On Mon, Aug 31, 2015 at 5:50 PM Duncan P. N. Exon Smith < >> >>> dexonsmith at apple.com <mailto:dexonsmith at apple.com>> wrote: >> >>>> >> >>>>> On 2015-Aug-31, at 12:21, Eric Christopher <echristo at gmail.com <mailto:echristo at gmail.com>> wrote: >> >>>>> Yep. This is where I was going :) >> >>>> >> >>>> Glad I found consensus, but I want to double-check that this makes >> >>>> sense to add to the driver. I didn't quite think through the >> >>>> implications myself. >> >>>> >> >>>> Since the driver doesn't know if there's any bitcode, or if LTO is >> >>>> going to be invoked, it seems like I'll have to change the noasserts >> >>>> driver to *always* pass the option to the linker just in case we are >> >>>> doing LTO. Is this reasonable? >> >>>> >> >>>> Also, I realized that passing `-mllvm -disable-llvm-verifier` to ld64 >> >>>> is redundant... so I'm thinking `-mllvm -disable-verify`. Make >> >>>> sense? >> >>>> >> >>>> *sigh* Reasons to hate the driver interface again... >> >>>> >> >>>> I guess this is ok. Could possibly add it to the existing terrible >> >>> "enable this pass" interface on liblto as well. >> >>> >> >>> The linker doesn't know whether clang was built with asserts, though. >> >>> >> >>> We could just make it implicit: move the decision to libLTO itself. Given >> >>> that clang and libLTO.dylib are different executables anyway -- and you >> >>> might be interposing an asserts libLTO.dylib to use with an installed clang >> >>> -- maybe it's even better? >> >>> >> >>> >> >> *nod* We could do that. Seems better than the alternative. >> > >> > +1 >> > >> >>>> I don't suppose ld64 could move to a model like we're talking about with >> >>> lld that pcc is working on? >> >>> >> >>> What specifically? >> >> >> >> >> >> Ah, using the C++ interface to handle everything and not using libLTO at >> >> all. >> >> >> >> He can speak more to this though. >> > >> > The C++ interface is much more convenient for a C++ program to use, but >> > clients need to revlock themselves to LLVM in order to use it. >> > >> > In theory the same does not apply to libLTO, but we do end up changing it >> > occasionally to add new APIs which ld64 promptly starts using, so it isn't >> > clear how much ld64 gains by relying on libLTO, >> >> Backward/Forward compatibility? >> Drop any version of clang/libLTO and still be able to use the system provided linker on any version of OS X? >> Sounds like a valuable feature to me, which is what I believe was (is?) sought by the C API in general (but that’s another story). >> >> >> Static linking against the parts of llvm you care about? There's nothing in the ld64 build system that means it can't do this :) > > I’m not sure how is that supposed to work? > I drop a new clang/libLTO on the system, clang generates bitcode, and libLTO handles it. > > If ld64 was statically linked against LLVM, it couldn’t read the new bitcode right? > > > Well sure, that's what Peter meant when he said revlocked. :)I’m not sure we’re talking about the same thing, I may have misunderstood his point (“In theory the same does not apply to libLTO, but we do end up changing it occasionally to add new APIs which ld64 promptly starts using, so it isn’t clear how much ld64 gains by relying on libLTO”) ; my take from that was that since the new ld64 starts using the new libLTO interface, there is a revlock anyway and then there is no advantage in keeping the old libLTO entry points. I just described a use case that shows how useful it can be. Apologize if I misunderstood the point. — Mehdi -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/43d28a77/attachment-0001.html>
Peter Collingbourne via llvm-dev
2015-Sep-04 18:38 UTC
[llvm-dev] RFC: LTO should use -disable-llvm-verifier
On Fri, Sep 04, 2015 at 11:13:43AM -0700, Mehdi Amini wrote:> > > On Sep 4, 2015, at 11:03 AM, Eric Christopher <echristo at gmail.com> wrote: > > > > > > > > On Fri, Sep 4, 2015 at 12:48 AM Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: > >> On Sep 4, 2015, at 12:22 AM, Eric Christopher <echristo at gmail.com <mailto:echristo at gmail.com>> wrote: > >> > >> > >> > >> On Thu, Sep 3, 2015 at 11:45 PM Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: > >> Hi, > >> > >> > On Sep 2, 2015, at 7:31 PM, Peter Collingbourne via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > >> > > >> > On Thu, Sep 03, 2015 at 01:10:42AM +0000, Eric Christopher wrote: > >> >> On Tue, Sep 1, 2015 at 10:43 AM Duncan P. N. Exon Smith < > >> >> dexonsmith at apple.com <mailto:dexonsmith at apple.com>> wrote: > >> >> > >> >>> > >> >>>> On 2015-Aug-31, at 18:09, Eric Christopher <echristo at gmail.com <mailto:echristo at gmail.com>> wrote: > >> >>>> > >> >>>> > >> >>>> > >> >>>> On Mon, Aug 31, 2015 at 5:50 PM Duncan P. N. Exon Smith < > >> >>> dexonsmith at apple.com <mailto:dexonsmith at apple.com>> wrote: > >> >>>> > >> >>>>> On 2015-Aug-31, at 12:21, Eric Christopher <echristo at gmail.com <mailto:echristo at gmail.com>> wrote: > >> >>>>> Yep. This is where I was going :) > >> >>>> > >> >>>> Glad I found consensus, but I want to double-check that this makes > >> >>>> sense to add to the driver. I didn't quite think through the > >> >>>> implications myself. > >> >>>> > >> >>>> Since the driver doesn't know if there's any bitcode, or if LTO is > >> >>>> going to be invoked, it seems like I'll have to change the noasserts > >> >>>> driver to *always* pass the option to the linker just in case we are > >> >>>> doing LTO. Is this reasonable? > >> >>>> > >> >>>> Also, I realized that passing `-mllvm -disable-llvm-verifier` to ld64 > >> >>>> is redundant... so I'm thinking `-mllvm -disable-verify`. Make > >> >>>> sense? > >> >>>> > >> >>>> *sigh* Reasons to hate the driver interface again... > >> >>>> > >> >>>> I guess this is ok. Could possibly add it to the existing terrible > >> >>> "enable this pass" interface on liblto as well. > >> >>> > >> >>> The linker doesn't know whether clang was built with asserts, though. > >> >>> > >> >>> We could just make it implicit: move the decision to libLTO itself. Given > >> >>> that clang and libLTO.dylib are different executables anyway -- and you > >> >>> might be interposing an asserts libLTO.dylib to use with an installed clang > >> >>> -- maybe it's even better? > >> >>> > >> >>> > >> >> *nod* We could do that. Seems better than the alternative. > >> > > >> > +1 > >> > > >> >>>> I don't suppose ld64 could move to a model like we're talking about with > >> >>> lld that pcc is working on? > >> >>> > >> >>> What specifically? > >> >> > >> >> > >> >> Ah, using the C++ interface to handle everything and not using libLTO at > >> >> all. > >> >> > >> >> He can speak more to this though. > >> > > >> > The C++ interface is much more convenient for a C++ program to use, but > >> > clients need to revlock themselves to LLVM in order to use it. > >> > > >> > In theory the same does not apply to libLTO, but we do end up changing it > >> > occasionally to add new APIs which ld64 promptly starts using, so it isn't > >> > clear how much ld64 gains by relying on libLTO, > >> > >> Backward/Forward compatibility? > >> Drop any version of clang/libLTO and still be able to use the system provided linker on any version of OS X? > >> Sounds like a valuable feature to me, which is what I believe was (is?) sought by the C API in general (but that’s another story). > >> > >> > >> Static linking against the parts of llvm you care about? There's nothing in the ld64 build system that means it can't do this :) > > > > I’m not sure how is that supposed to work? > > I drop a new clang/libLTO on the system, clang generates bitcode, and libLTO handles it. > > > > If ld64 was statically linked against LLVM, it couldn’t read the new bitcode right? > > > > > > Well sure, that's what Peter meant when he said revlocked. :) > > I’m not sure we’re talking about the same thing, I may have misunderstood his point (“In theory the same does not apply to libLTO, but we do end up changing it > occasionally to add new APIs which ld64 promptly starts using, so it isn’t clear how much ld64 gains by relying on libLTO”) ; my take from that was that since the new ld64 starts using the new libLTO interface, there is a revlock anyway and then there is no advantage in keeping the old libLTO entry points. > I just described a use case that shows how useful it can be. > > Apologize if I misunderstood the point.That seems like a potentially valid use case, but if it doesn't actually work right now it doesn't really buy you much. If this is indeed an abolute requirement, there are better ways of achieving this kind of thing without creating an API stability burden in LLVM. For example you could make libLTO a build product of ld64 rather than of LLVM by copying the tools/lto code into ld64 and using the preprocessor to make the code LLVM-version-dependent. This seems like it could be relatively easily achievable right now; all you'd need to do is switch the Xcode toolchain over to shipping the libLTO build product from ld64 instead of from LLVM (or even statically linking as Eric suggested). Any other existing clients of libLTO who care about this can do the same thing. Another advantage of doing this is that you can change the "stable" libLTO API without breaking clients, as ld64 would be the only client. Thanks, -- Peter