Sean Fertile via llvm-dev
2017-Jun-14 04:43 UTC
[llvm-dev] Refining which symbols are preemptable with lto
As a follow up to https://reviews.llvm.org/D20217 I would like to use lto/thinlto to refine when a symbol is marked as local/preemptable. I'm not very familiar with lto though so would appreciate some guidance about how best to go about this. Regards Sean Fertile
Peter Collingbourne via llvm-dev
2017-Jun-14 06:03 UTC
[llvm-dev] Refining which symbols are preemptable with lto
On Tue, Jun 13, 2017 at 9:43 PM, Sean Fertile via llvm-dev < llvm-dev at lists.llvm.org> wrote:> As a follow up to https://reviews.llvm.org/D20217 I would like to use > lto/thinlto to refine when a symbol is marked as local/preemptable. > I'm not very familiar with lto though so would appreciate some > guidance about how best to go about this. >Hi Sean, The first thing that I would do is to try to get this working for regular LTO. There is a flag in the lto::SymbolResolution class named FinalDefinitionInLinkageUnit, which was intended to control whether the backend must make symbol references preemptible. Right now we copy a number of other fields from SymbolResolution onto the symbol definition, which is done by LTO::addRegularLTO in this loop: https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/LTO/LTO.cpp#L555 I imagine that this is where you would add code that translates FinalDefinitionInLinkageUnit into dso_local/dso_preemptible. The next step would be to get it working for ThinLTO. This would involve the same basic mechanism: the FinalDefinitionInLinkageUnit flag would control whether to set dso_local/dso_preemptible on global values. The difference would be that the flag would need to be passed via the module summary index, as this is how information needs to be passed to the ThinLTO backends. The way that I imagine that this would work is that we would introduce a new flag in GlobalValueSummary::GVFlags into which you would copy the value of FinalDefinitionInLinkageUnit during LTO::addThinLTO. We already copy the Prevailing flag elsewhere using the GUID: https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/LTO/LTO.cpp#L578 but for this flag I imagine that you would want to copy it directly into the global's summary (see ModuleSummaryIndex::getValueInfo for how to look up a global value summary using a GUID). The default value of this flag should mean "preemptible" -- Apple's linker uses a legacy API which will not set this flag, so that avoids breaking their linker. Then in the thinBackend function you would translate the flag into dso_local/dso_preemptible. Probably the best place to do that is in FunctionImportGlobalProcessing::processGlobalForThinLTO here: https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/Transforms/Utils/FunctionImportUtils.cpp#L215 See shouldPromoteLocalToGlobal for how to find the summary for the GlobalVariable at that point -- some of that code probably needs to be factored out. There would also need to be bitcode reader/writer support for the new flag: https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/Bitcode/Reader/BitcodeReader.cpp#L865 https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp#L879 There is one subtlety however, which is that the summary index might not contain entries for functions without definitions. For now I would punt on this and mark declarations that are not present in the summary index as preemptible. Peter> > > Regards > Sean Fertile > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-- -- Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170613/394063b1/attachment.html>
Sean Fertile via llvm-dev
2017-Jun-14 15:36 UTC
[llvm-dev] Refining which symbols are preemptable with lto
Thank you for such detailed instructions Peter! On Wed, Jun 14, 2017 at 2:03 AM, Peter Collingbourne <peter at pcc.me.uk> wrote:> On Tue, Jun 13, 2017 at 9:43 PM, Sean Fertile via llvm-dev > <llvm-dev at lists.llvm.org> wrote: >> >> As a follow up to https://reviews.llvm.org/D20217 I would like to use >> lto/thinlto to refine when a symbol is marked as local/preemptable. >> I'm not very familiar with lto though so would appreciate some >> guidance about how best to go about this. > > > Hi Sean, > > The first thing that I would do is to try to get this working for regular > LTO. There is a flag in the lto::SymbolResolution class named > FinalDefinitionInLinkageUnit, which was intended to control whether the > backend must make symbol references preemptible. Right now we copy a number > of other fields from SymbolResolution onto the symbol definition, which is > done by LTO::addRegularLTO in this loop: > https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/LTO/LTO.cpp#L555 > I imagine that this is where you would add code that translates > FinalDefinitionInLinkageUnit into dso_local/dso_preemptible. > > The next step would be to get it working for ThinLTO. This would involve the > same basic mechanism: the FinalDefinitionInLinkageUnit flag would control > whether to set dso_local/dso_preemptible on global values. The difference > would be that the flag would need to be passed via the module summary index, > as this is how information needs to be passed to the ThinLTO backends. The > way that I imagine that this would work is that we would introduce a new > flag in GlobalValueSummary::GVFlags into which you would copy the value of > FinalDefinitionInLinkageUnit during LTO::addThinLTO. We already copy the > Prevailing flag elsewhere using the GUID: > https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/LTO/LTO.cpp#L578 > but for this flag I imagine that you would want to copy it directly into the > global's summary (see ModuleSummaryIndex::getValueInfo for how to look up a > global value summary using a GUID). > > The default value of this flag should mean "preemptible" -- Apple's linker > uses a legacy API which will not set this flag, so that avoids breaking > their linker. > > Then in the thinBackend function you would translate the flag into > dso_local/dso_preemptible. Probably the best place to do that is in > FunctionImportGlobalProcessing::processGlobalForThinLTO here: > https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/Transforms/Utils/FunctionImportUtils.cpp#L215 > See shouldPromoteLocalToGlobal for how to find the summary for the > GlobalVariable at that point -- some of that code probably needs to be > factored out. > > There would also need to be bitcode reader/writer support for the new flag: > https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/Bitcode/Reader/BitcodeReader.cpp#L865 > https://github.com/llvm-project/llvm-project-20170507/blob/4240587b0d490325dae8aa64d620ce599fa20840/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp#L879 > There is one subtlety however, which is that the summary index might not > contain entries for functions without definitions. For now I would punt on > this and mark declarations that are not present in the summary index as > preemptible. > > Peter >> >> >> >> Regards >> Sean Fertile >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > > -- > -- > Peter