llvm dev - Oct 2013 - [LLVMdev] [Debug Info + LTO] Type Uniquing for C types?

On Fri, Oct 11, 2013 at 12:40 PM, Eric Christopher <echristo at
gmail.com>wrote:
> >> It depends upon the goals. If the goal is to make debug
information
> >> post-link smaller then just using the type hashing machinery for
> >> structs will be sufficient.
> >
> >
> > By "the type hashing machinery for structs", are you
referring to the
> type
> > hashing at the back end?
> >
>
> I am, yes, since there's no other place we do currently.
>
> >>
> >> However, if it's to save space during an
> >> LTO link then we'll want to do it in the front end.
> >
> >
> > Yes, my purpose here is to save memory space in number of MDNodes
(also
> # of
> > DIEs) generated in a LTO build.
> > Type hashing at the DIE level can reduce the dwarf size.
> >
>
> I agree with both of these statements.
>
> I also agree with the desire to help LTO memory consumption so we'll
> need something from the front end for this since we'd like to continue
> to use the folding set to do the uniquing.
>
Hi Eric,

Assume that we need to do type hashing (i.e. assume Doug's rules for
merging C types do not apply), would you
like it to be done on AST types or IR debug info metadata types?

One possibility is to hash the IR debug info types in DIBuilder::finalize.
When we are creating the MD nodes, we can provide a simple identifier that
is unique within the DIBuider. In finailize(),
we update the identifier to include a hash value (prepend the type name),
so types constructed by one DIBuilder will be unique
from types constructed by another DIBUilder unless they are structurally
equivalent by having the same hash.

We can then use the folding set to do the uniquing across CUs.

Thanks,
Manman

>
> -eric
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131014/ef8672f9/attachment.html>

On Mon, Oct 14, 2013 at 1:08 PM, Manman Ren <manman.ren at gmail.com>
wrote:
>
>
>
> On Fri, Oct 11, 2013 at 12:40 PM, Eric Christopher <echristo at
gmail.com>wrote:
>
>> >> It depends upon the goals. If the goal is to make debug
information
>> >> post-link smaller then just using the type hashing machinery
for
>> >> structs will be sufficient.
>> >
>> >
>> > By "the type hashing machinery for structs", are you
referring to the
>> type
>> > hashing at the back end?
>> >
>>
>> I am, yes, since there's no other place we do currently.
>>
>> >>
>> >> However, if it's to save space during an
>> >> LTO link then we'll want to do it in the front end.
>> >
>> >
>> > Yes, my purpose here is to save memory space in number of MDNodes
(also
>> # of
>> > DIEs) generated in a LTO build.
>> > Type hashing at the DIE level can reduce the dwarf size.
>> >
>>
>> I agree with both of these statements.
>>
>> I also agree with the desire to help LTO memory consumption so
we'll
>> need something from the front end for this since we'd like to
continue
>> to use the folding set to do the uniquing.
>>
>
> Hi Eric,
>
> Assume that we need to do type hashing (i.e. assume Doug's rules for
> merging C types do not apply),
>
Now the assumption is true, any opinion on where to do the hashing?

Thanks,
Manman

would you like it to be done on AST types or IR debug info metadata
types?
>
> One possibility is to hash the IR debug info types in DIBuilder::finalize.
> When we are creating the MD nodes, we can provide a simple identifier that
> is unique within the DIBuider. In finailize(),
> we update the identifier to include a hash value (prepend the type name),
> so types constructed by one DIBuilder will be unique
> from types constructed by another DIBUilder unless they are structurally
> equivalent by having the same hash.
>
> We can then use the folding set to do the uniquing across CUs.
>
> Thanks,
> Manman
>
>
>>
>> -eric
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131014/992d5958/attachment.html>

On Mon, Oct 14, 2013 at 4:14 PM, Manman Ren <manman.ren at gmail.com>
wrote:
>
>
>
> On Mon, Oct 14, 2013 at 1:08 PM, Manman Ren <manman.ren at gmail.com>
wrote:
>>
>>
>>
>>
>> On Fri, Oct 11, 2013 at 12:40 PM, Eric Christopher <echristo at
gmail.com>
>> wrote:
>>>
>>> >> It depends upon the goals. If the goal is to make debug
information
>>> >> post-link smaller then just using the type hashing
machinery for
>>> >> structs will be sufficient.
>>> >
>>> >
>>> > By "the type hashing machinery for structs", are you
referring to the
>>> > type
>>> > hashing at the back end?
>>> >
>>>
>>> I am, yes, since there's no other place we do currently.
>>>
>>> >>
>>> >> However, if it's to save space during an
>>> >> LTO link then we'll want to do it in the front end.
>>> >
>>> >
>>> > Yes, my purpose here is to save memory space in number of
MDNodes (also
>>> > # of
>>> > DIEs) generated in a LTO build.
>>> > Type hashing at the DIE level can reduce the dwarf size.
>>> >
>>>
>>> I agree with both of these statements.
>>>
>>> I also agree with the desire to help LTO memory consumption so
we'll
>>> need something from the front end for this since we'd like to
continue
>>> to use the folding set to do the uniquing.
>>
>>
>> Hi Eric,
>>
>> Assume that we need to do type hashing (i.e. assume Doug's rules
for
>> merging C types do not apply),
>
>
> Now the assumption is true, any opinion on where to do the hashing?
>
We should still do it in the front end for the types with a language
specific way. Nothing has greatly changed versus, say, C++ here - it's
just easier in C++ because of the language.

-eric