Chris Lattner
2013-Nov-13 02:02 UTC
[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)
On Nov 12, 2013, at 4:59 PM, Chandler Carruth <chandlerc at google.com> wrote:> Other options are: > 1> Using a different LLVMContext for the destination module, but it didn’t work out since Linker was not designed to work with different LLVMContexts for source vs destination. > 2> removeUnusedMDNodes checks if a MDNode is used in a different way (i.e use_empty() && !hasValueHandler()), but it does not remove MDNodes that form cycles. > > 3) Make the MDNode be owned by the module that uses it? > > MDNode is shared among modules so multiple modules can use it, if we specify an owner for a MDNode, that will prevent sharing. > > From your stats (40% stuck in the old module) it doesn't sound like this is buying us anything... >If the old module is deleted, then these MDNodes can be reclaimed. I think this proposal amounts to a “garbage collector” that clears out now-dead IR objects that are uniqued in the LLVM Context. While MDNodes are your focus, the same thing would apply equally well to ConstantInt and other things that may become unreachable. Details matter on this (for it to be efficient), but I think that it would be very useful for LLVMContext to have a method that goes through and releases IR objects that aren’t used. This could be used by the LTO driver, and if it actually reduces memory by 40%, that would be huge. -Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/734dfb66/attachment.html>
Chandler Carruth
2013-Nov-13 02:07 UTC
[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)
On Tue, Nov 12, 2013 at 6:02 PM, Chris Lattner <clattner at apple.com> wrote:> On Nov 12, 2013, at 4:59 PM, Chandler Carruth <chandlerc at google.com> > wrote: > > Other options are: >>>> 1> Using a different LLVMContext for the destination module, but it >>>> didn’t work out since Linker was not designed to work with different >>>> LLVMContexts for source vs destination. >>>> 2> removeUnusedMDNodes checks if a MDNode is used in a different way >>>> (i.e use_empty() && !hasValueHandler()), but it does not remove MDNodes >>>> that form cycles. >>>> >>> >>> 3) Make the MDNode be owned by the module that uses it? >>> >> >> MDNode is shared among modules so multiple modules can use it, if we >> specify an owner for a MDNode, that will prevent sharing. >> > > From your stats (40% stuck in the old module) it doesn't sound like this > is buying us anything... > > > > If the old module is deleted, then these MDNodes can be reclaimed. > > I think this proposal amounts to a “garbage collector” that clears out > now-dead IR objects that are uniqued in the LLVM Context. While MDNodes > are your focus, the same thing would apply equally well to ConstantInt and > other things that may become unreachable. >I think the big difference is that there appears to be very little (relatively speaking) overlap between modules for MDNodes. This is different from types and many other things I suspect. My question was: if we're not seeing a higher degree of sharing between modules by using context-uniqued MDNodes, why not have them be owned by the module rather than the context?> > Details matter on this (for it to be efficient), but I think that it would > be very useful for LLVMContext to have a method that goes through and > releases IR objects that aren’t used. This could be used by the LTO > driver, and if it actually reduces memory by 40%, that would be huge. >My only worry is doing a relatively expensive walk over all types and constants when most are actually shared and don't get deleted. It sounds like the metadata nodes are weird in that they are mostly disjoint between modules. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/b4ef29e5/attachment.html>
Chris Lattner
2013-Nov-13 05:17 UTC
[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)
On Nov 12, 2013, at 6:07 PM, Chandler Carruth <chandlerc at google.com> wrote:> My question was: if we're not seeing a higher degree of sharing between modules by using context-uniqued MDNodes, why not have them be owned by the module rather than the context?Wouldn't that force the IR linker to *copy* the MDNodes from the source module into the dest module when linking? -Chris
Seemingly Similar Threads
- [LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)
- [LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)
- [LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)
- [LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)
- [LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)