[llvm-dev] RFC: LTO should use -disable-llvm-verifier

Mehdi Amini via llvm-dev
Fri, 04 Sep 2015 22:11:39 -0000

> On Sep 4, 2015, at 11:38 AM, Peter Collingbourne <peter at pcc.me.uk>
wrote:
>
> On Fri, Sep 04, 2015 at 11:13:43AM -0700, Mehdi Amini wrote:
>>
>>> On Sep 4, 2015, at 11:03 AM, Eric Christopher <echristo at
gmail.com> wrote:
>>>
>>>
>>>
>>> On Fri, Sep 4, 2015 at 12:48 AM Mehdi Amini <mehdi.amini at
apple.com <mailto:mehdi.amini at apple.com>> wrote:
>>>> On Sep 4, 2015, at 12:22 AM, Eric Christopher <echristo at
gmail.com <mailto:echristo at gmail.com>> wrote:
>>>>
>>>>
>>>>
>>>> On Thu, Sep 3, 2015 at 11:45 PM Mehdi Amini <mehdi.amini at
apple.com <mailto:mehdi.amini at apple.com>> wrote:
>>>> Hi,
>>>>
>>>>> On Sep 2, 2015, at 7:31 PM, Peter Collingbourne via
llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>>>>>
>>>>> On Thu, Sep 03, 2015 at 01:10:42AM +0000, Eric Christopher
wrote:
>>>>>> On Tue, Sep 1, 2015 at 10:43 AM Duncan P. N. Exon Smith
<
>>>>>> dexonsmith at apple.com <mailto:dexonsmith at
apple.com>> wrote:
>>>>>>
>>>>>>>
>>>>>>>> On 2015-Aug-31, at 18:09, Eric Christopher
<echristo at gmail.com <mailto:echristo at gmail.com>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 31, 2015 at 5:50 PM Duncan P. N.
Exon Smith <
>>>>>>> dexonsmith at apple.com <mailto:dexonsmith at
apple.com>> wrote:
>>>>>>>>
>>>>>>>>> On 2015-Aug-31, at 12:21, Eric Christopher
<echristo at gmail.com <mailto:echristo at gmail.com>> wrote:
>>>>>>>>> Yep. This is where I was going :)
>>>>>>>>
>>>>>>>> Glad I found consensus, but I want to
double-check that this makes
>>>>>>>> sense to add to the driver.  I didn't quite
think through the
>>>>>>>> implications myself.
>>>>>>>>
>>>>>>>> Since the driver doesn't know if
there's any bitcode, or if LTO is
>>>>>>>> going to be invoked, it seems like I'll
have to change the noasserts
>>>>>>>> driver to *always* pass the option to the
linker just in case we are
>>>>>>>> doing LTO.  Is this reasonable?
>>>>>>>>
>>>>>>>> Also, I realized that passing `-mllvm
-disable-llvm-verifier` to ld64
>>>>>>>> is redundant... so I'm thinking `-mllvm
-disable-verify`.  Make
>>>>>>>> sense?
>>>>>>>>
>>>>>>>> *sigh* Reasons to hate the driver interface
again...
>>>>>>>>
>>>>>>>> I guess this is ok. Could possibly add it to
the existing terrible
>>>>>>> "enable this pass" interface on liblto as
well.
>>>>>>>
>>>>>>> The linker doesn't know whether clang was built
with asserts, though.
>>>>>>>
>>>>>>> We could just make it implicit: move the decision
to libLTO itself.  Given
>>>>>>> that clang and libLTO.dylib are different
executables anyway -- and you
>>>>>>> might be interposing an asserts libLTO.dylib to use
with an installed clang
>>>>>>> -- maybe it's even better?
>>>>>>>
>>>>>>>
>>>>>> *nod* We could do that. Seems better than the
alternative.
>>>>>
>>>>> +1
>>>>>
>>>>>>>> I don't suppose ld64 could move to a model
like we're talking about with
>>>>>>> lld that pcc is working on?
>>>>>>>
>>>>>>> What specifically?
>>>>>>
>>>>>>
>>>>>> Ah, using the C++ interface to handle everything and
not using libLTO at
>>>>>> all.
>>>>>>
>>>>>> He can speak more to this though.
>>>>>
>>>>> The C++ interface is much more convenient for a C++ program
to use, but
>>>>> clients need to revlock themselves to LLVM in order to use
it.
>>>>>
>>>>> In theory the same does not apply to libLTO, but we do end
up changing it
>>>>> occasionally to add new APIs which ld64 promptly starts
using, so it isn't
>>>>> clear how much ld64 gains by relying on libLTO,
>>>>
>>>> Backward/Forward compatibility?
>>>> Drop any version of clang/libLTO and still be able to use the
system provided linker on any version of OS X?
>>>> Sounds like a valuable feature to me, which is what I believe
was (is?) sought by the C API in general (but that’s another story).
>>>>
>>>>
>>>> Static linking against the parts of llvm you care about?
There's nothing in the ld64 build system that means it can't do this :)
>>>
>>> I’m not sure how is that supposed to work?
>>> I drop a new clang/libLTO on the system, clang generates bitcode,
and libLTO handles it.
>>>
>>> If ld64 was statically linked against LLVM, it couldn’t read the
new bitcode right?
>>>
>>>
>>> Well sure, that's what Peter meant when he said revlocked. :)
>>
>> I’m not sure we’re talking about the same thing, I may have
misunderstood his point (“In theory the same does not apply to libLTO, but we do
end up changing it
>> occasionally to add new APIs which ld64 promptly starts using, so it
isn’t clear how much ld64 gains by relying on libLTO”) ; my take from that was
that since the new ld64 starts using the new libLTO interface, there is a
revlock anyway and then there is no advantage in keeping the old libLTO entry
points.
>> I just described a use case that shows how useful it can be.
>>
>> Apologize if I misunderstood the point.
>
> That seems like a potentially valid use case, but if it doesn't
actually
> work right now it doesn't really buy you much.

It is working right now AFAIK. At least my understand is that the current
situation is that you can drop a clang/libLTO built today on a machine that has
ld64 from a few years ago and it will work.


>
> If this is indeed an abolute requirement, there are better ways of
achieving
> this kind of thing without creating an API stability burden in LLVM. For
> example you could make libLTO a build product of ld64 rather than of LLVM
by
> copying the tools/lto code into ld64 and using the preprocessor to make the
> code LLVM-version-dependent. This seems like it could be relatively easily
> achievable right now; all you'd need to do is switch the Xcode
toolchain
> over to shipping the libLTO build product from ld64 instead of from LLVM
> (or even statically linking as Eric suggested). Any other existing clients
> of libLTO who care about this can do the same thing.

It seems I’m missing something: the system ld64 linker will be shipped with a
self-contained libLTO, but the new compiler I’ll drop-in will generate
“new" bitcode that will be fed to this “old” libLTO, and that’s not gonna
play well.
Unless you are only including the C API wrapper in ld64, but in this case I’m
not sure I understand "using the preprocessor to make the code
LLVM-version-dependent”. You will be able to build ld64 independently of LLVM,
but how about at runtime? The compiler needs to ship a dynamic library that will
be loaded by the linker, and there is still a need for a stable API there.

—
Mehdi


> Another advantage of doing this is that you can change the
"stable" libLTO
> API without breaking clients, as ld64 would be the only client.
>
> Thanks,
> --
> Peter