> On May 4, 2017, at 4:53 PM, David Blaikie <dblaikie at gmail.com> wrote: > > Alrighty, a little fuzzy on how best to implement this - Adrian, you've probably got the most context here as to how to wrangle this. > > My first attempt was in IRMover.cpp, IRLinker::linkFunctionBody - after metadata is copied over, create a new subprogram derived from Dst.getSubprogram, only changing the CU over (to the first, if any, CU on the llvm.dbg.cu <http://llvm.dbg.cu/> named metadata). > > This is insufficient because all the metadata in the function (variables, etc) doesn't get updated and chains up to the old DISubprogram still.For this part you should be able to reuse the code I'm writing for ConeFunction right now (that can re-parent a function's debug info inside a new DISubprogram). Hopefully I'll have something out for review tomorrow.> > Next attempt was to RAUW the foreign CU with the desired CU, but that fails with "MDNode::replaceAllUsesWith - expected temporary node"Non-temporary MDNodes don't have RAUW support (this was deemed too expensive). You either need to create them pointing to a temporary node and then rauw the temp node with a distinct node, or you need tot create them already pointing to the right CU. Thankfully only DISubprograms should point to a CU node. -- adrian> So - any ideas? > > On Thu, May 4, 2017 at 7:40 AM Teresa Johnson <tejohnson at google.com <mailto:tejohnson at google.com>> wrote: > On Thu, May 4, 2017 at 7:22 AM, Rafael Avila de Espindola via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > David Blaikie via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> writes: > > > So Dehao and I have been dealing with some of the nitty gritty details of > > debug info with ThinLTO, specifically with Fission(Split DWARF). > > > > This applies to LTO as well, so I won't single out ThinLTO here. > > When are the .dwo files produced? If at link time, what is the advantage > of using fission? Normally the advantage is the faster link, but if it > is the linker producing them, what is left? > > For ThinLTO they are produced by the backends, and only the split .o part needs to be linked. I think that benefit would occur with in-process ThinLTO as well (the .o files handed back to the linker are smaller), but for us where we have a distributed ThinLTO build, the backends are producing split .o and .dwo files and so the final native link process gets all the normal advantages of fission. > > For regular LTO, I suppose the same benefit could occur - the backend produces a split .o and .dwo file and only passes the split .o file to the link. > > Teresa > > > Cheers, > Rafael > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com <mailto:tejohnson at google.com> | 408-460-2413 <tel:(408)%20460-2413>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170504/c2d87995/attachment.html>
On Thu, May 4, 2017 at 5:01 PM Adrian Prantl <aprantl at apple.com> wrote:> On May 4, 2017, at 4:53 PM, David Blaikie <dblaikie at gmail.com> wrote: > > Alrighty, a little fuzzy on how best to implement this - Adrian, you've > probably got the most context here as to how to wrangle this. > > My first attempt was in IRMover.cpp, IRLinker::linkFunctionBody - after > metadata is copied over, create a new subprogram derived from > Dst.getSubprogram, only changing the CU over (to the first, if any, CU on > the llvm.dbg.cu named metadata). > > This is insufficient because all the metadata in the function (variables, > etc) doesn't get updated and chains up to the old DISubprogram still. > > > For this part you should be able to reuse the code I'm writing for > ConeFunction right now (that can re-parent a function's debug info inside a > new DISubprogram). Hopefully I'll have something out for review tomorrow. >How fortuitous - I'll keep an eye out for it!> > > Next attempt was to RAUW the foreign CU with the desired CU, but that > fails with "MDNode::replaceAllUsesWith - expected temporary node" > > > Non-temporary MDNodes don't have RAUW support (this was deemed too > expensive). You either need to create them pointing to a temporary node and > then rauw the temp node with a distinct node, or you need tot create them > already pointing to the right CU. Thankfully only DISubprograms should > point to a CU node. > > -- adrian > > So - any ideas? > > On Thu, May 4, 2017 at 7:40 AM Teresa Johnson <tejohnson at google.com> > wrote: > >> On Thu, May 4, 2017 at 7:22 AM, Rafael Avila de Espindola via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> David Blaikie via llvm-dev <llvm-dev at lists.llvm.org> writes: >>> >>> > So Dehao and I have been dealing with some of the nitty gritty details >>> of >>> > debug info with ThinLTO, specifically with Fission(Split DWARF). >>> > >>> > This applies to LTO as well, so I won't single out ThinLTO here. >>> >>> When are the .dwo files produced? If at link time, what is the advantage >>> of using fission? Normally the advantage is the faster link, but if it >>> is the linker producing them, what is left? >>> >> >> For ThinLTO they are produced by the backends, and only the split .o part >> needs to be linked. I think that benefit would occur with in-process >> ThinLTO as well (the .o files handed back to the linker are smaller), but >> for us where we have a distributed ThinLTO build, the backends are >> producing split .o and .dwo files and so the final native link process gets >> all the normal advantages of fission. >> >> For regular LTO, I suppose the same benefit could occur - the backend >> produces a split .o and .dwo file and only passes the split .o file to the >> link. >> >> Teresa >> >> >>> Cheers, >>> Rafael >> >> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >> >> >> >> -- >> Teresa Johnson | Software Engineer | tejohnson at google.com | >> 408-460-2413 <(408)%20460-2413> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170505/20e30769/attachment-0001.html>
After a couple of different attemps/approaches, I actually came up with a more surgical fix than modifying the LTO steps Committed in r302858 - it's a fix down in the DWARF emission layer (lib/Codegen/AsmPrinter) to avoid the cross-CU references but otherwise keep everything in the CU it came from (so this'll make normal LTO sane - most stuff will still go in the CU it was written in and DIEs will be duplicated only when needed). It also keeps the in-.o inlining info that can be emitted with Fission using the ref_addr as well. & there's a flag to get the old behavior, for those using Fission without DWP files or testing DWP improvements. On Thu, May 4, 2017 at 5:32 PM David Blaikie <dblaikie at gmail.com> wrote:> On Thu, May 4, 2017 at 5:01 PM Adrian Prantl <aprantl at apple.com> wrote: > >> On May 4, 2017, at 4:53 PM, David Blaikie <dblaikie at gmail.com> wrote: >> >> Alrighty, a little fuzzy on how best to implement this - Adrian, you've >> probably got the most context here as to how to wrangle this. >> >> My first attempt was in IRMover.cpp, IRLinker::linkFunctionBody - after >> metadata is copied over, create a new subprogram derived from >> Dst.getSubprogram, only changing the CU over (to the first, if any, CU on >> the llvm.dbg.cu named metadata). >> >> This is insufficient because all the metadata in the function (variables, >> etc) doesn't get updated and chains up to the old DISubprogram still. >> >> >> For this part you should be able to reuse the code I'm writing for >> ConeFunction right now (that can re-parent a function's debug info inside a >> new DISubprogram). Hopefully I'll have something out for review tomorrow. >> > > How fortuitous - I'll keep an eye out for it! > > >> >> >> Next attempt was to RAUW the foreign CU with the desired CU, but that >> fails with "MDNode::replaceAllUsesWith - expected temporary node" >> >> >> Non-temporary MDNodes don't have RAUW support (this was deemed too >> expensive). You either need to create them pointing to a temporary node and >> then rauw the temp node with a distinct node, or you need tot create them >> already pointing to the right CU. Thankfully only DISubprograms should >> point to a CU node. >> >> -- adrian >> >> So - any ideas? >> >> On Thu, May 4, 2017 at 7:40 AM Teresa Johnson <tejohnson at google.com> >> wrote: >> >>> On Thu, May 4, 2017 at 7:22 AM, Rafael Avila de Espindola via llvm-dev < >>> llvm-dev at lists.llvm.org> wrote: >>> >>>> David Blaikie via llvm-dev <llvm-dev at lists.llvm.org> writes: >>>> >>>> > So Dehao and I have been dealing with some of the nitty gritty >>>> details of >>>> > debug info with ThinLTO, specifically with Fission(Split DWARF). >>>> > >>>> > This applies to LTO as well, so I won't single out ThinLTO here. >>>> >>>> When are the .dwo files produced? If at link time, what is the advantage >>>> of using fission? Normally the advantage is the faster link, but if it >>>> is the linker producing them, what is left? >>>> >>> >>> For ThinLTO they are produced by the backends, and only the split .o >>> part needs to be linked. I think that benefit would occur with in-process >>> ThinLTO as well (the .o files handed back to the linker are smaller), but >>> for us where we have a distributed ThinLTO build, the backends are >>> producing split .o and .dwo files and so the final native link process gets >>> all the normal advantages of fission. >>> >>> For regular LTO, I suppose the same benefit could occur - the backend >>> produces a split .o and .dwo file and only passes the split .o file to the >>> link. >>> >>> Teresa >>> >>> >>>> Cheers, >>>> Rafael >>> >>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>> >>> >>> >>> -- >>> Teresa Johnson | Software Engineer | tejohnson at google.com | >>> 408-460-2413 <(408)%20460-2413> >>> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170512/219e9b58/attachment.html>