similar to: [LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)

On Tue, Nov 12, 2013 at 4:38 PM, Chandler Carruth <chandlerc at google.com>wrote: > > On Tue, Nov 12, 2013 at 4:29 PM, Manman Ren <manman.ren at gmail.com> wrote: > >> Hi All, >> >> In LTO, we load in the source modules and link the source modules into a >> destination module. >> Lots of MDNodes are only used by the source modules, for example

On Tue, Nov 12, 2013 at 4:29 PM, Manman Ren <manman.ren at gmail.com> wrote: > Hi All, > > In LTO, we load in the source modules and link the source modules into a > destination module. > Lots of MDNodes are only used by the source modules, for example Xalan > used 649MB for MDNodes after loading and linking, but the actual > destination module only has 393MB of

On Tue, Nov 12, 2013 at 4:46 PM, Manman Ren <manman.ren at gmail.com> wrote: > > > > On Tue, Nov 12, 2013 at 4:38 PM, Chandler Carruth <chandlerc at google.com>wrote: > >> >> On Tue, Nov 12, 2013 at 4:29 PM, Manman Ren <manman.ren at gmail.com> wrote: >> >>> Hi All, >>> >>> In LTO, we load in the source modules and

On Tue, Nov 12, 2013 at 4:59 PM, Chandler Carruth <chandlerc at google.com>wrote: > On Tue, Nov 12, 2013 at 4:46 PM, Manman Ren <manman.ren at gmail.com> wrote: > >> >> >> >> On Tue, Nov 12, 2013 at 4:38 PM, Chandler Carruth <chandlerc at google.com>wrote: >> >>> >>> On Tue, Nov 12, 2013 at 4:29 PM, Manman Ren <manman.ren

On Nov 12, 2013, at 4:59 PM, Chandler Carruth <chandlerc at google.com> wrote: > Other options are: > 1> Using a different LLVMContext for the destination module, but it didn’t work out since Linker was not designed to work with different LLVMContexts for source vs destination. > 2> removeUnusedMDNodes checks if a MDNode is used in a different way (i.e use_empty() &&

On Tue, Nov 12, 2013 at 6:02 PM, Chris Lattner <clattner at apple.com> wrote: > On Nov 12, 2013, at 4:59 PM, Chandler Carruth <chandlerc at google.com> > wrote: > > Other options are: >>>> 1> Using a different LLVMContext for the destination module, but it >>>> didn’t work out since Linker was not designed to work with different >>>>

On Thu, Nov 14, 2013 at 5:19 AM, Rafael Espíndola <rafael.espindola at gmail.com> wrote: >> Letting the module own MDNodes may not be a win for gold since it is going >> to create multiple copies of MDNodes that could be shared with Context >> owning MDNodes. >> >> For example, with debug info type uniquing, the type nodes can be shared >> across modules,

> Letting the module own MDNodes may not be a win for gold since it is going > to create multiple copies of MDNodes that could be shared with Context > owning MDNodes. > > For example, with debug info type uniquing, the type nodes can be shared > across modules, but with module owning MDNodes, each module will create its > own copy of the type nodes. The advantage is that the

On Wed, Nov 13, 2013 at 7:58 PM, Rafael Espíndola < rafael.espindola at gmail.com> wrote: > > It sounds like the linker could call lto_module_dispose() right after > > lto_codegen_add_module() to help reduce the memory footprint. That > would be > > a simple linker change. A slightly larger linker change would be to > > immediately call lto_codegen_add_module()

On Nov 12, 2013, at 6:11 PM, Chandler Carruth <chandlerc at google.com> wrote: > On Tue, Nov 12, 2013 at 6:07 PM, Manman Ren <manman.ren at gmail.com> wrote: > Hi Chandler, > > I don't quite get why you think sharing is not buying us anything... > It reduces the memory footprint of the source modules (there is sharing among the source modules) and the number of

On Tue, Nov 12, 2013 at 6:07 PM, Manman Ren <manman.ren at gmail.com> wrote: > Hi Chandler, > > I don't quite get why you think sharing is not buying us anything... > It reduces the memory footprint of the source modules (there is sharing > among the source modules) and the number of MDNodes created for the > destination module (we do not need to re-create the MDNodes

On Jun 20, 2013, at 3:27 PM, David Blaikie wrote: > +1 to roughly everything Eric said. > > On Thu, Jun 20, 2013 at 2:51 PM, Manman Ren <mren at apple.com> wrote: >> >> The intent of this proposal is to speedup compilation of "-flto -g" for c++ programs. >> This is based on discussions with Adrian, David and Eric. >> >>

+1 to roughly everything Eric said. On Thu, Jun 20, 2013 at 2:51 PM, Manman Ren <mren at apple.com> wrote: > > The intent of this proposal is to speedup compilation of "-flto -g" for c++ programs. > This is based on discussions with Adrian, David and Eric. > > --------------------------- > Problem: > A single class can be used in multiple source files and the

The intent of this proposal is to speedup compilation of "-flto -g" for c++ programs. This is based on discussions with Adrian, David and Eric. --------------------------- Problem: A single class can be used in multiple source files and the DI (Debug Info) class is included in multiple bc files. The duplication of class definitions causes blow-up in # of MDNodes, # of DIEs, leading to

> It sounds like the linker could call lto_module_dispose() right after > lto_codegen_add_module() to help reduce the memory footprint. That would be > a simple linker change. A slightly larger linker change would be to > immediately call lto_codegen_add_module() right after > lto_module_create_from_memory(), then lto_module_dispose(). That is, never > have any unmerged

On Jun 20, 2013, at 2:58 PM, Eric Christopher wrote: > Hi Manman, > > On Thu, Jun 20, 2013 at 2:51 PM, Manman Ren <mren at apple.com> wrote: >> >> The intent of this proposal is to speedup compilation of "-flto -g" for c++ programs. >> This is based on discussions with Adrian, David and Eric. >> > > Thanks for bringing this back to the

On Thu, Jun 20, 2013 at 4:45 PM, Manman Ren <mren at apple.com> wrote: > > On Jun 20, 2013, at 3:55 PM, Manman Ren <mren at apple.com> wrote: > > > On Jun 20, 2013, at 2:58 PM, Eric Christopher wrote: > > Hi Manman, > > On Thu, Jun 20, 2013 at 2:51 PM, Manman Ren <mren at apple.com> wrote: > > > The intent of this proposal is to speedup

On Thu, Jun 20, 2013 at 5:13 PM, Manman Ren <mren at apple.com> wrote: > > On Jun 20, 2013, at 4:52 PM, David Blaikie wrote: > >> On Thu, Jun 20, 2013 at 4:45 PM, Manman Ren <mren at apple.com> wrote: >>> >>> On Jun 20, 2013, at 3:55 PM, Manman Ren <mren at apple.com> wrote: >>> >>> >>> On Jun 20, 2013, at 2:58 PM, Eric

On Jun 20, 2013, at 4:52 PM, David Blaikie wrote: > On Thu, Jun 20, 2013 at 4:45 PM, Manman Ren <mren at apple.com> wrote: >> >> On Jun 20, 2013, at 3:55 PM, Manman Ren <mren at apple.com> wrote: >> >> >> On Jun 20, 2013, at 2:58 PM, Eric Christopher wrote: >> >> Hi Manman, >> >> On Thu, Jun 20, 2013 at 2:51 PM, Manman Ren

Hi Manman (and llvmdev), I filed these two bugs to track the ideas that I was cooking: http://llvm.org/bugs/show_bug.cgi?id=17891 http://llvm.org/bugs/show_bug.cgi?id=17892 TL;DR: I'm saying we should go from: !14 = metadata !{i32 786445, metadata !1, metadata !10, metadata !"y", i32 3, i64 32, i64 32, i64 32, i32 0, metadata !13} to: !14 = metadata