I sent a patch to llvm-commits. You can see the code at
http://reviews.llvm.org/D10036. Thanks!
On Sun, May 24, 2015 at 4:40 PM, Rui Ueyama <ruiu at google.com> wrote:
> I'm sorry for not updating the thread -- I thought I did that before.
>
> I started experimenting the idea by implementing a minimal linker using
> the section-based design with some additional simplification/optimizations.
> It's already able to link small programs like the LLD itself, and the
> performance looks indeed better (probably the LLD is too small as a
> benchmark, but the new one is more than 2x faster). I also believe that the
> code is more readable than the current COFF port.
>
> I've just hacked it up, so it needs time to clean up. I think I can
send a
> patch for review this week.
>
>
> On Tue, May 12, 2015 at 12:38 PM, Lang Hames <lhames at gmail.com>
wrote:
>
>> Hi Rui,
>>
>> I'd like to preserve the native format, and I'm happy to own
it. I'm
>> still getting up to speed on LLD though, so it may take me a little
while
>> to improve the tooling/testing for it.
>>
>> Cheers,
>> Lang.
>>
>>
>> On Thu, May 7, 2015 at 1:28 PM, Rui Ueyama <ruiu at google.com>
wrote:
>>
>>> On Thu, May 7, 2015 at 12:58 PM, Jim Grosbach <grosbach at
apple.com>
>>> wrote:
>>>
>>>> Hi Rui,
>>>>
>>>> Thank you for clarifying. This is very helpful.
>>>>
>>>> It’s unfortunate that you’re not seeing benefits from the
increased
>>>> semantic knowledge the atom based model can provide. I know
you’ve explored
>>>> the issue thoroughly, though, so I understand why you’re
wanting to move a
>>>> different direction for your platform.
>>>>
>>>> It’s reasonable to me to split the logic along atom based vs.
section
>>>> based in the LLD codebase. Ideally I’d love for that not to be
so, but the
>>>> practical results indicate that it is. I agree there is still
worthwhile
>>>> code sharing that can and should be done between the two. We’re
talking
>>>> about expanding the LLD project’s scope to include multiple
linking models,
>>>> not forking the project.
>>>>
>>>> It will be important to keep the layering here such that the
linking
>>>> model choice is orthogonal to the file format choice. It should
be possible
>>>> to construct both an atom based ELF linker and a section based
Mach-O
>>>> linker, for example, even though the default choice for both
formats is the
>>>> other way around. That way different platforms with different
constraints,
>>>> such as what Alex talked about earlier, can make the choice of
model and
>>>> the choice of representation independently.
>>>>
>>>> As a second step, I would very much like to see the native
format
>>>> brought back, if only for the atom-based model. Do you feel
this is doable?
>>>>
>>>
>>> Yes, it's doable, but I'd really like to see this back with
different
>>> unit tests because the way the feature was tested was unrealistic
and hard
>>> to maintain. Previously, we tested the feature by dumping
intermediate
>>> linker state to a Native file and reading it back from a file to
resume
>>> processing. That was different from the expected use case, which is
to use
>>> Native files as an alternative object file format. Creating a
checkpoint
>>> file of the linker state is a different feature (and I guess nobody
really
>>> intended to implement such feature.)
>>>
>>> I think we need to extend yaml2obj tool to write outputs in the
Native
>>> format, and use the tool to feed Native object files to the linker.
>>>
>>> Is there anyone who wants to own the feature? Or I could just
revive the
>>> code, but I'd really want to avoid doing that if that
doesn't come with a
>>> good test...
>>>
>>>
>>>> -Jim
>>>>
>>>> On May 6, 2015, at 2:18 PM, Rui Ueyama <ruiu at
google.com> wrote:
>>>>
>>>> I'm sorry if my suggestion gave an impression that I
disregard the
>>>> Mach-O port of the LLD linker. I do care about Mach-O. I do not
plan to
>>>> break or remove any functionality from the current Mach-O port
of the LLD.
>>>> I don't propose to remove the atom model from the linker as
long as it
>>>> seems to be a good fit for the port (and looks like it is).
>>>>
>>>> As to the proposal to have two different linkers, I'd think
that that's
>>>> not really a counter-proposal, as it's similar to what
I'm proposing.
>>>>
>>>> Maybe the view of "future file formats vs the existing
formats" (or
>>>> "experimental platform vs. practical tool") is not
right to get the
>>>> difference between the atom model and the section model, since
the Mach-O
>>>> file an existing file format which we'd want to keep to be
on the atom
>>>> model. I think we want both even for the existing formats.
>>>>
>>>> My proposal can be read as suggesting we split the LLD linker
into two
>>>> major parts, the atom model-based and the section model-based,
while
>>>> keeping the two under the same project and repository. I still
think that
>>>> we can share code between the two, especially for the LTO,
which is I
>>>> prefer to have the two under the same repository.
>>>>
>>>> On Mon, May 4, 2015 at 12:52 PM, Chris Lattner <clattner at
apple.com>
>>>> wrote:
>>>>
>>>>> On May 1, 2015, at 12:31 PM, Rui Ueyama <ruiu at
google.com> wrote:
>>>>>
>>>>> *Proposal*
>>>>>
>>>>> 1. Re-architect the linker based on the section model
where it’s
>>>>> appropriate.
>>>>> 2. Stop simulating different linker semantics using the
Unix
>>>>> model. Instead, directly implement the native behavior.
>>>>>
>>>>> Preface: I have never personally contributed code to LLD,
so don’t
>>>>> take anything I’m about to say too seriously. This is not
a mandate or
>>>>> anything, just an observation/idea.
>>>>>
>>>>>
>>>>> I think that there is an alternative solution to these
exact same
>>>>> problems. What you’ve identified here is that there are
two camps of
>>>>> people working on LLD, and they have conflicting goals:
>>>>>
>>>>> - Camp A: LLD is infrastructure for the next generation of
awesome
>>>>> linking and toolchain features, it should take advantage of
how compilers
>>>>> work to offer new features, performance, etc without deep
concern for
>>>>> compatibility.
>>>>>
>>>>> - Camp B: LLD is a drop in replacement system linker
(notably for COFF
>>>>> and ELF systems), which is best of breed and with no
compromises w.r.t.
>>>>> that goal.
>>>>>
>>>>>
>>>>> I think the problem here is that these lead to natural and
inescapable
>>>>> tensions, and Alex summarized how Camp B has been steering
LLD away from
>>>>> what Camp A people want. This isn’t bad in and of itself,
because what
>>>>> Camp B wants is clearly and unarguably good for LLVM.
However, it is also
>>>>> not sufficient, and while innovation in the linker space
(e.g. a new
>>>>> “native” object file format generated directly from
compiler structures)
>>>>> may or may not actually “work” or be “worth it”, we won’t
know unless we
>>>>> try, and that won’t fulfill its promise if there are
compromises to Camp B.
>>>>>
>>>>> So here’s my counterproposal: *two different linkers.*
>>>>>
>>>>> Lets stop thinking about lld as one linker, and instead
think of it is
>>>>> two different ones. We’ll build a Camp B linker which is
the best of breed
>>>>> section based linker. It will support linker scripts and
do everything
>>>>> better than any existing section based linker. The first
step of this is
>>>>> to do what Rui proposes and rip atoms out of the model.
>>>>>
>>>>> We will *also* build a no-holds-barred awesome atom based
linker that
>>>>> takes advantage of everything it can from LLVM’s
architecture to enable
>>>>> innovative new tools without worrying too much about
backwards
>>>>> compatibility.
>>>>>
>>>>> These two linkers should share whatever code makes sense,
but also
>>>>> shouldn’t try to share code that doesn’t make sense. The
split between the
>>>>> semantic model of sections vs atoms seems like a very
natural one to me.
>>>>>
>>>>> One question is: does it make sense for these to live in
the same lld
>>>>> subproject, or be split into two different subprojects? I
think the answer
>>>>> to that question is driven from whether there is shared
code common between
>>>>> the two linkers that doesn’t make sense to sink down to the
llvm subproject
>>>>> itself.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> -Chris
>>>>>
>>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150526/7e1cf8de/attachment.html>