llvm dev - Jun 2018 - Mach-O support in lld: what are the known issues?

Thanks for the response, Rui!

On Tue, Jun 5, 2018 at 5:26 PM, Rui Ueyama <ruiu at google.com>
wrote:
>
> Besides the features you pointed out, I think Xcode introduced a new way
> of listing dynamic linking symbols, and I believe lld doesn't support
that.
>
.tbd files, is that right? A colleague of mine pointed me to Apple's
libtapi open source project [1], maybe I can learn more about these files
from that library. In fact, there's been discussion about bringing libtapi
to LLVM on the mailing list in the past, although I don't know if anything
came of it. [2]

I think the real issue is the lack of maintenance and ownership of
the
> mach-O lld tree. There's no activities for the tree for years, though
we've
> been making efforts to keep it compile and pass all the existing tests.
>
Excellent, thanks for letting me know. This doesn't bother me, I'm happy
to
try contributing to it as best I can! I would also appreciate, as your time
permits, whatever guidance you can provide. For example, benefitting from
several years of hindsight, would you recommend keeping the ATOM-based lld
approach? [3] Prior emails discussed moving Mach-O lld away from ATOM. [4]
Has the success of ELF and COFF influenced your thinking on this in the
years since, or is ATOM probably still the best fit for Mach-O?

On Wed, Jun 6, 2018 at 8:10 PM, Jean-Daniel via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
> The list of feature exclusive to the Apple platform and Mac-O in the
> linker is astonishingly long. I started listing the missing features in lld
> at some points, but stop midway.
>
Jean-Daniel, do you still happen to have this list somewhere? I'd love to
read it, even if you did stop midway! Besides .tbd files, for example
there's the matter of supporting "fat binaries," which contain
code for
multiple instruction sets.

Also, thanks to everyone on the thread who expressed interest! I'm still
exploring and don't want to commit to anything just yet, but suffice it to
say I'm interested in lld Mach-O as well.

- Brian Gesiak

[1] https://opensource.apple.com/source/tapi/tapi-1.30/
[2] https://lists.llvm.org/pipermail/llvm-dev/2017-September/117264.html
[3] https://lld.llvm.org/AtomLLD.html
[4] http://lists.llvm.org/pipermail/llvm-dev/2015-May/085115.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180606/82825160/attachment.html>

On Wed, Jun 6, 2018 at 11:41 AM Brian Gesiak <modocache at gmail.com>
wrote:
> Thanks for the response, Rui!
>
> On Tue, Jun 5, 2018 at 5:26 PM, Rui Ueyama <ruiu at google.com>
wrote:
>>
>> Besides the features you pointed out, I think Xcode introduced a new
way
>> of listing dynamic linking symbols, and I believe lld doesn't
support that.
>>
>
> .tbd files, is that right? A colleague of mine pointed me to Apple's
> libtapi open source project [1], maybe I can learn more about these files
> from that library. In fact, there's been discussion about bringing
libtapi
> to LLVM on the mailing list in the past, although I don't know if
anything
> came of it. [2]
>
> I think the real issue is the lack of maintenance and ownership of the
>> mach-O lld tree. There's no activities for the tree for years,
though we've
>> been making efforts to keep it compile and pass all the existing tests.
>>
>
> Excellent, thanks for letting me know. This doesn't bother me, I'm
happy
> to try contributing to it as best I can! I would also appreciate, as your
> time permits, whatever guidance you can provide. For example, benefitting
> from several years of hindsight, would you recommend keeping the ATOM-based
> lld approach? [3] Prior emails discussed moving Mach-O lld away from ATOM.
> [4] Has the success of ELF and COFF influenced your thinking on this in the
> years since, or is ATOM probably still the best fit for Mach-O?
>
That's a good question. There was a big discussion as to the design of the
new (now current ELF/COFF/wasm) and the ATOM-based lld a few years ago when
I started working on the new one. At the time no one including me was
really sure what design is desirable, and I was exploring the design space
to something good. Today, we have three working, high-performance linkers
for ELF, COFF and wasm based on the new design, which I think proves the
design; it is easy to add new features, easy to understand, and it delivers
what users want the most (i.e. speed). Given that, if I were you, I'd try
to see if the new lld's design fits mach-O. You may need to tweak the
design a little bit, but I'd imagine that the difference is not as
significant as between ELF and wasm (which has a different concept of
memory address space mainly for security). I'd also like to get input from
Apple engineers as well.

On Wed, Jun 6, 2018 at 8:10 PM, Jean-Daniel via llvm-dev
<
> llvm-dev at lists.llvm.org> wrote:
>>
>> The list of feature exclusive to the Apple platform and Mac-O in the
>> linker is astonishingly long. I started listing the missing features in
lld
>> at some points, but stop midway.
>>
>
> Jean-Daniel, do you still happen to have this list somewhere? I'd love
to
> read it, even if you did stop midway! Besides .tbd files, for example
> there's the matter of supporting "fat binaries," which
contain code for
> multiple instruction sets.
>
> Also, thanks to everyone on the thread who expressed interest! I'm
still
> exploring and don't want to commit to anything just yet, but suffice it
to
> say I'm interested in lld Mach-O as well.
>
> - Brian Gesiak
>
> [1] https://opensource.apple.com/source/tapi/tapi-1.30/
> [2] https://lists.llvm.org/pipermail/llvm-dev/2017-September/117264.html
> [3] https://lld.llvm.org/AtomLLD.html
> [4] http://lists.llvm.org/pipermail/llvm-dev/2015-May/085115.html
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180606/104c624c/attachment.html>

> Le 6 juin 2018 à 22:37, Rui Ueyama <ruiu at google.com> a écrit :
> 
> On Wed, Jun 6, 2018 at 11:41 AM Brian Gesiak <modocache at gmail.com
<mailto:modocache at gmail.com>> wrote:
> Thanks for the response, Rui!
> 
> On Tue, Jun 5, 2018 at 5:26 PM, Rui Ueyama <ruiu at google.com
<mailto:ruiu at google.com>> wrote:
> Besides the features you pointed out, I think Xcode introduced a new way of
listing dynamic linking symbols, and I believe lld doesn't support that.
> 
> .tbd files, is that right? A colleague of mine pointed me to Apple's
libtapi open source project [1], maybe I can learn more about these files from
that library. In fact, there's been discussion about bringing libtapi to
LLVM on the mailing list in the past, although I don't know if anything came
of it. [2]
> 
> I think the real issue is the lack of maintenance and ownership of the
mach-O lld tree. There's no activities for the tree for years, though
we've been making efforts to keep it compile and pass all the existing
tests.
> 
> Excellent, thanks for letting me know. This doesn't bother me, I'm
happy to try contributing to it as best I can! I would also appreciate, as your
time permits, whatever guidance you can provide. For example, benefitting from
several years of hindsight, would you recommend keeping the ATOM-based lld
approach? [3] Prior emails discussed moving Mach-O lld away from ATOM. [4] Has
the success of ELF and COFF influenced your thinking on this in the years since,
or is ATOM probably still the best fit for Mach-O?
> 
> That's a good question. There was a big discussion as to the design of
the new (now current ELF/COFF/wasm) and the ATOM-based lld a few years ago when
I started working on the new one. At the time no one including me was really
sure what design is desirable, and I was exploring the design space to something
good. Today, we have three working, high-performance linkers for ELF, COFF and
wasm based on the new design, which I think proves the design; it is easy to add
new features, easy to understand, and it delivers what users want the most (i.e.
speed). Given that, if I were you, I'd try to see if the new lld's
design fits mach-O. You may need to tweak the design a little bit, but I'd
imagine that the difference is not as significant as between ELF and wasm (which
has a different concept of memory address space mainly for security). I'd
also like to get input from Apple engineers as well.
I don’t remember but I think one of the main point was that EFL and Mach-O don’t
have the same concept of section. IIRC, the concept of section in ELF was closer
to the atom model than with Mach-O which uses few sections and put a lot of
things in each one.

And a quick search gave me that: 
http://llvm.1065342.n5.nabble.com/LLD-improvement-plan-td80788.html#a80871
<http://llvm.1065342.n5.nabble.com/LLD-improvement-plan-td80788.html#a80871>




-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180606/7bb0355b/attachment.html>