Xinliang David Li
2015-Jul-21 22:35 UTC
[LLVMdev] RFC: ThinLTO Symbol Linkage and Renaming
> > This scares me a little for linkonce -- there's a minor change to > semantics if the importing module would have linked against a > *different* definition of the same symbol -- but I'm not really > sure it matters much. > >This should not be an issue in practice as it exists non thinLTO compilations too. For instance changing the optimization level of one module can lead to different inline decisions and different copy of the comdat function to be picked up in the end.> > > > I wonder whether a prefix would be better? > > All the promotion/renaming scares me. I feel like there may be > dragons here we're not aware of. > >The promotion scheme is similar to LIPO which has been exercised on large number of huge C++ apps -- I think it is robust.> The only concrete concern I have (once you switch to hidden > visibility) is the interaction with non-LTO'd objects being linked > into the same executable, but I wonder if I'm missing something > else, too... > >There is an limitation (also with LIPO, LTO) that you can not compile one module say 'a.c' in thinLTO mode with the rest of the modules, and later recompile 'a.c' into 'a.o' without thinLTO and tries to mix and match with the rest of the real object files built with thinLTO unless 'a.c' is not imported by any other modules or it does not have any statics to be promoted. thanks, David> > > > 5 Linkage Change Summary Table > > > > The following table summarizes the linkage change that happen during > ThinLTO backend compilations. Note that the only time the linkage changes > in the original module is for the static promotion case (for Internal and > Private linkages), where it changes to External as described in Section > 4.3.1. In all other cases the linkage in the original module stays the > same, and therefore is not noted in the table below. > > > > Note that there are no InternalLinkage, PrivateLinkage, > AvailableExternallyLinkage, LinkOnce*Linkage, Weak*Linkage, > AppendingLinkage and CommonLinkage declarations. There are no > ExternalWeakLinkage definitions. > > > > > +---------------------------------+------------------------------------------------+ > > | | Importing Module Linkage > | > > | Original Module > +----------------------+-------------------------+ > > | | Import definition | Import as > declaration | > > > +----------------------------------------------------------------------------------+ > > | External (def) | AvailableExternally | External > | > > > +----------------------------------------------------------------------------------+ > > | External (decl) | N/A | External > | > > > +----------------+-----------------------------------------------------------------+ > > | | Promote[1] | AvailableExternally | External > | > > | Internal (def) > +-----------------------------------------------------------------+ > > | | NoPromote | Internal | N/A (force > import def) | > > > +----------------------------------------------------------------------------------+ > > | | Promote[1] | AvailableExternally | External > | > > | Private (def) > +-----------------------------------------------------------------+ > > | | NoPromote | Private | N/A (force > import def) | > > > +----------------+-----------------------------------------------------------------+ > > | AvailableExternally (def) | AvailableExternally | External > | > > > +----------------------------------------------------------------------------------+ > > | LinkOnceAny (def) | LinkOnceAny | N/A (force > import def) | > > > +----------------------------------------------------------------------------------+ > > | LinkOnceODR (def) | LinkOnceODR | N/A (force > import def) | > > > +----------------------------------------------------------------------------------+ > > | WeakAny (def) | N/A (never import) | > ExternalWeak | > > > +----------------------------------------------------------------------------------+ > > | WeakODR (def) | WeakODR | > ExternalWeak | > > > +----------------------------------------------------------------------------------+ > > | Appending (def) (only variables)| N/A (never import) | N/A (never > import) | > > > +----------------------------------------------------------------------------------+ > > | ExternalWeak (decl) | N/A | > ExternalWeak | > > > +----------------------------------------------------------------------------------+ > > | Common (def) (only variables) | Common | N/A (always > import def)| > > > +---------------------------------+----------------------+-------------------------+ > > > > [1] Linkage in original module changes to External > > > > -- > > Teresa Johnson | Software Engineer | tejohnson at google.com | > 408-460-2413 > > > <ThinLTOSymbolLinkageandRenaming.pdf>_______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150721/357930e0/attachment.html>
Duncan P. N. Exon Smith
2015-Jul-27 20:06 UTC
[LLVMdev] RFC: ThinLTO Symbol Linkage and Renaming
> On 2015-Jul-21, at 15:35, Xinliang David Li <xinliangli at gmail.com> wrote: > > > > This scares me a little for linkonce -- there's a minor change to > semantics if the importing module would have linked against a > *different* definition of the same symbol -- but I'm not really > sure it matters much. > > > > This should not be an issue in practice as it exists non thinLTO compilations too. For instance changing the optimization level of one module can lead to different inline decisions and different copy of the comdat function to be picked up in the end. >Good point.> > > > > I wonder whether a prefix would be better? > > All the promotion/renaming scares me. I feel like there may be > dragons here we're not aware of. > > > The promotion scheme is similar to LIPO which has been exercised on large number of huge C++ apps -- I think it is robust.Sure, but other languages, platforms and conventions may yield other problems. Anyway, it's just a theoretical concern, I don't have anything concrete!> > The only concrete concern I have (once you switch to hidden > visibility) is the interaction with non-LTO'd objects being linked > into the same executable, but I wonder if I'm missing something > else, too... > > > There is an limitation (also with LIPO, LTO) that you can not compile one module say 'a.c' in thinLTO mode with the rest of the modules, and later recompile 'a.c' into 'a.o' without thinLTO and tries to mix and match with the rest of the real object files built with thinLTO unless 'a.c' is not imported by any other modules or it does not have any statics to be promoted.Interesting. I don't think LTO has this limitation. But I also don't think this is too important; if someone turns off thin-LTO for a particular object file, the build system should detect that the object changed and regenerate all the things that depend on it, so there's no correctness problem here; and it seems like a rare thing to toggle, so I can't see this really causing a concern for compile time.
Xinliang David Li
2015-Jul-27 20:29 UTC
[LLVMdev] RFC: ThinLTO Symbol Linkage and Renaming
On Mon, Jul 27, 2015 at 1:06 PM, Duncan P. N. Exon Smith < dexonsmith at apple.com> wrote:> > > On 2015-Jul-21, at 15:35, Xinliang David Li <xinliangli at gmail.com> > wrote: > > > > > > > > This scares me a little for linkonce -- there's a minor change to > > semantics if the importing module would have linked against a > > *different* definition of the same symbol -- but I'm not really > > sure it matters much. > > > > > > > > This should not be an issue in practice as it exists non thinLTO > compilations too. For instance changing the optimization level of one > module can lead to different inline decisions and different copy of the > comdat function to be picked up in the end. > > > > Good point. > > > > > > > > > > I wonder whether a prefix would be better? > > > > All the promotion/renaming scares me. I feel like there may be > > dragons here we're not aware of. > > > > > > The promotion scheme is similar to LIPO which has been exercised on > large number of huge C++ apps -- I think it is robust. > > Sure, but other languages, platforms and conventions may yield > other problems. Anyway, it's just a theoretical concern, I don't > have anything concrete! > > > > > The only concrete concern I have (once you switch to hidden > > visibility) is the interaction with non-LTO'd objects being linked > > into the same executable, but I wonder if I'm missing something > > else, too... > > > > > > There is an limitation (also with LIPO, LTO) that you can not compile > one module say 'a.c' in thinLTO mode with the rest of the modules, and > later recompile 'a.c' into 'a.o' without thinLTO and tries to mix and match > with the rest of the real object files built with thinLTO unless 'a.c' is > not imported by any other modules or it does not have any statics to be > promoted. > > Interesting. I don't think LTO has this limitation. But I also > don't think this is too important; if someone turns off thin-LTO for > a particular object file, the build system should detect that the > object changed and regenerate all the things that depend on it, so > there's no correctness problem here; and it seems like a rare thing > to toggle, so I can't see this really causing a concern for compile > time.Its main usage I can see is for bug triaging (using object mix&match) -- there are other ways to do that. For instance 1) add option to force static promotion for O2 or selectively disabling importing etc. So right, it is only a concern on paper. thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150727/0beefdd5/attachment.html>