On Mon, May 04, 2015 at 12:52:55PM -0700, Chris Lattner wrote:> I think the problem here is that these lead to natural and inescapable > tensions, and Alex summarized how Camp B has been steering LLD away > from what Camp A people want. This isn’t bad in and of itself, because > what Camp B wants is clearly and unarguably good for LLVM. However, > it is also not sufficient, and while innovation in the linker space > (e.g. a new “native” object file format generated directly from > compiler structures) may or may not actually “work” or be “worth it”, > we won’t know unless we try, and that won’t fulfill its promise if > there are compromises to Camp B.It has been said in this thread before, but I fail to see how the atom model is an actual improvement over the fine grained section model. It seems to be artifically restricted for no good reasons.> Lets stop thinking about lld as one linker, and instead think of it is > two different ones. We’ll build a Camp B linker which is the best of > breed section based linker. It will support linker scripts and do > everything better than any existing section based linker. The first > step of this is to do what Rui proposes and rip atoms out of the model.This is another item that has been irritating me. While it is a very laudable goal to not depend on linker scripts for the common case, not having the functionality of fine grained output control is certainly a problem. They are crucial for embedded developers and also at least significant for anything near a system kernel.> We will also build a no-holds-barred awesome atom based linker that > takes advantage of everything it can from LLVM’s architecture to enable > innovative new tools without worrying too much about backwards > compatibility.I'd say that a good justificatiton for way an atom based linker is/can be better would be a good start... Joerg
On May 4, 2015, at 1:16 PM, Joerg Sonnenberger <joerg at britannica.bec.de> wrote:> It has been said in this thread before, but I fail to see how the atom > model is an actual improvement over the fine grained section model. It > seems to be artifically restricted for no good reasons.Sections come with a huge amount of bloat and overhead that atoms do not.>> Lets stop thinking about lld as one linker, and instead think of it is >> two different ones. We’ll build a Camp B linker which is the best of >> breed section based linker. It will support linker scripts and do >> everything better than any existing section based linker. The first >> step of this is to do what Rui proposes and rip atoms out of the model. > > This is another item that has been irritating me. While it is a very > laudable goal to not depend on linker scripts for the common case, not > having the functionality of fine grained output control is certainly a > problem. They are crucial for embedded developers and also at least > significant for anything near a system kernel.I’m not saying that the linker should eschew fine grained control, I’m saying it should dump linker scripts (and replace them with something better). Are you going to argue that linker scripts are great, or that they are what we would end up with if we weren’t driven by backwards compatibility goals? -Chris
On Mon, May 4, 2015 at 3:08 PM Chris Lattner <clattner at apple.com> wrote:> On May 4, 2015, at 1:16 PM, Joerg Sonnenberger <joerg at britannica.bec.de> > wrote: > > It has been said in this thread before, but I fail to see how the atom > > model is an actual improvement over the fine grained section model. It > > seems to be artifically restricted for no good reasons. > > Sections come with a huge amount of bloat and overhead that atoms do not. > >Caveat: My response isn't in favor of one way over the other, but providing some explanations or trying to frame the trade-offs in a more explicit form. Sections come with _some_ size overhead that an atom doesn't because an atom is contained within a section, however, atoms come with a link time cost that sections don't. Trade off. :) The atom model is effective for minimizing the size of on disk representation or working around limited object container formats. Being able to just append sections is a trade off for speed of linking versus size of inputs at the moment (which is a tricky trade-off as the speed of linking starts to approach the size of the objects being linked). That said, I don't think any of the current linkers are currently I/O bound here either.> >> Lets stop thinking about lld as one linker, and instead think of it is > >> two different ones. We’ll build a Camp B linker which is the best of > >> breed section based linker. It will support linker scripts and do > >> everything better than any existing section based linker. The first > >> step of this is to do what Rui proposes and rip atoms out of the model. > > > > This is another item that has been irritating me. While it is a very > > laudable goal to not depend on linker scripts for the common case, not > > having the functionality of fine grained output control is certainly a > > problem. They are crucial for embedded developers and also at least > > significant for anything near a system kernel. > > I’m not saying that the linker should eschew fine grained control, I’m > saying it should dump linker scripts (and replace them with something > better). Are you going to argue that linker scripts are great, or that > they are what we would end up with if we weren’t driven by backwards > compatibility goals? >Linker scripts are worse than everything - except for the alternatives that we know about. Any particular suggestions here? -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150504/9896dd99/attachment.html>
On Mon, May 04, 2015 at 03:05:47PM -0700, Chris Lattner wrote:> On May 4, 2015, at 1:16 PM, Joerg Sonnenberger <joerg at britannica.bec.de> wrote: > > It has been said in this thread before, but I fail to see how the atom > > model is an actual improvement over the fine grained section model. It > > seems to be artifically restricted for no good reasons. > > Sections come with a huge amount of bloat and overhead that atoms do not.I don't buy that as far as the internal representation is concerned. On-disk format as Eric said is a somewhat different question.> >> Lets stop thinking about lld as one linker, and instead think of it is > >> two different ones. We’ll build a Camp B linker which is the best of > >> breed section based linker. It will support linker scripts and do > >> everything better than any existing section based linker. The first > >> step of this is to do what Rui proposes and rip atoms out of the model. > > > > This is another item that has been irritating me. While it is a very > > laudable goal to not depend on linker scripts for the common case, not > > having the functionality of fine grained output control is certainly a > > problem. They are crucial for embedded developers and also at least > > significant for anything near a system kernel. > > I’m not saying that the linker should eschew fine grained control, I’m > saying it should dump linker scripts (and replace them with something > better). Are you going to argue that linker scripts are great, or that > they are what we would end up with if we weren’t driven by backwards > compatibility goals?I haven't seen the better alternative yet. It is hard to reason about vaporware. I'm not particulary attached to linker scripts, they are certainly a horrible language. But the lack of support is certainly a show stopper for those areas where the functionality is needed. How is a linker supporting at least a major part of that functionality going to be different from a linker that accepts linker scripts as input? Joerg
Replying to the thread, not just the email since I was on vacations. First, note that there is a nomenclature issue. A section in ELF/COFF is closer to an atom in MachO than a MachO section IMHO. A rose by any other name would smell as sweet, but sure as hell creates a lot of confusion :-) On 4 May 2015 at 18:05, Chris Lattner <clattner at apple.com> wrote:> On May 4, 2015, at 1:16 PM, Joerg Sonnenberger <joerg at britannica.bec.de> wrote: >> It has been said in this thread before, but I fail to see how the atom >> model is an actual improvement over the fine grained section model. It >> seems to be artifically restricted for no good reasons. > > Sections come with a huge amount of bloat and overhead that atoms do not.No, they don't. Not on ELF for sure. On ELF a section is just a entry into a table marking a region in the file. The "huge amount of bloat" that people associate with sections is actually just the extra space for the ultra large section names ".text._ZFoo....". Create multiple sections with the same name (I implemented that) and the bloat goes away. As has been pointed before, a section in ELF is just a better version of what is called an Atom in lld: It is a chunk of the file that the linker can move, but it also supports multiple symbols, which is handy for things like making the C1 and C2 constructors share the same address or how MSVC implement vtables+rtti. Atoms being a distinct entity from sections (i.e., having non atomic sections) is a necessity for MachO because it has more restrictive sections (as Kevin was kind enough to explain). Another way of looking at it (for understanding, I wouldn't use the nomenclature in code) is that with this proposal lld will still be atom based, we will just be extending atoms to support multiple symbols. The logic for splitting sections into atoms would become * ELF/COFF: one atom per section. * MachO: One atom per global symbol. And so MachO ends up with atoms that have only one symbol, but that is just a special case.>> This is another item that has been irritating me. While it is a very >> laudable goal to not depend on linker scripts for the common case, not >> having the functionality of fine grained output control is certainly a >> problem. They are crucial for embedded developers and also at least >> significant for anything near a system kernel. > > I’m not saying that the linker should eschew fine grained control, I’m saying it should dump linker scripts (and replace them with something better). Are you going to argue that linker scripts are great, or that they are what we would end up with if we weren’t driven by backwards compatibility goals?I agree that this is a distinct issue. Linker scripts are a backward compatibility pain. Directly using sections for ELF/COFF is *better* than what is currently being done in lld. As for organization, I agree with Rui's suggestion of 2 linkers in one. One is ELF/COFF and uses sections, one is MachO and uses atoms. Even with the split there is still enough common code that I don't think having two repositories would help. I don't agree that there is value in keeping the current atom on top ELF/COFF. It just adds cost to two formats whose sections are already flexible atoms. It also prevents optimizations like not even reading duplicated comdats. Last but not least, on the idea of a new object format: Everyone that has worked on linkers or assemblers has a list of things they don't like about the format that was being used (I do for sure). It is entirely possible that if we get our thoughts together we can build a better format. Having said that, an object file format has a tremendous cost. Just look at the pain that is maintaining support for mips' interpretation of r_info. We have to be sure there is a genuine advantage to it before adding a new object format to the world. To know that I think we need to push the current formats to see how far they go. As an analogy, imagine if people working on BFD had decided that ELF linking was too slow or missing features and had decided to create a new format that fit BFD better. That would have been really unfortunate, because as gold showed the problem was not ELF, it was the organization of BFD, but now we would probably be stuck supporting 4 formats in llvm and lld. Once we have a linker (and MC) that is as good as it gets for ELF/COFF and MachO we well be in a good position for discussing a new format. Cheers, Rafael