llvm dev - Oct 2014 - [LLVMdev] [RFC] Less memory and greater maintainability for debug info IR

On 15/10/2014, 17:31 , Eric Christopher wrote:
> On Wed, Oct 15, 2014 at 2:30 PM, Sean Silva <chisophugis at
gmail.com> wrote:
>> Running `llvm-link *.bc` would OOM a machine with 64GB of RAM (with -g;
>> without -g it completed with much less). The increasing could be easily
>> watched on the system "process monitor" in real time.
>>
> This is likely what we've already discussed and was handled a long
> while ago now.
Wait, really? I can definitely get my 64Gb box to thrash just trying to 
llvm-link -g bitcode files. By 'handled' you mean fixed in trunk or 
'plan to fix'?


Diego.

On Fri, Oct 17, 2014 at 7:27 AM, Diego Novillo <dnovillo at google.com>
wrote:
>
> On 15/10/2014, 17:31 , Eric Christopher wrote:
>>
>> On Wed, Oct 15, 2014 at 2:30 PM, Sean Silva <chisophugis at
gmail.com> wrote:
>>>
>>> Running `llvm-link *.bc` would OOM a machine with 64GB of RAM (with
-g;
>>> without -g it completed with much less). The increasing could be
easily
>>> watched on the system "process monitor" in real time.
>>>
>> This is likely what we've already discussed and was handled a long
>> while ago now.
>
>
> Wait, really? I can definitely get my 64Gb box to thrash just trying to
> llvm-link -g bitcode files. By 'handled' you mean fixed in trunk or
'plan to
> fix'?
>
Fixed in trunk. You're dealing with an order of magnitude more source
than most of these people.

Basically there's the problems highlighted in my talk at Euro2013
(that you even attended ;)

http://llvm.org/devmtg/2013-04/christopher-slides.pdf

that were fixed in the intervening year and we updated here:

http://llvm.org/devmtg/2013-11/slides/Christopher-DebugInfo.pdf

See the stuff about type merging. You don't even want to think about
how bad it would be otherwise - just think about "why modules" and
imagine including every header into every .cpp file :)

Now we're dealing with:

a) the inefficiencies of the way that we use metadata to encode dwarf and,
b) the inefficiencies of metadata in general

This is probably going to be worth quite a bit, but it's the next step
in dealing with debug info size.

Also keep in mind that even in a fully linked binary debug info is
_big_ - this is dealing with the fact that our representation of debug
info in IR has never really been optimized and is currently designed
for maximum flexibility. Same with metadata really, we're just dealing
with the scaling problems of metadata here.

-eric

On Fri, Oct 17, 2014 at 1:21 PM, Eric Christopher <echristo at gmail.com>
wrote:

See the stuff about type merging. You don't even want to think
about
> how bad it would be otherwise - just think about "why modules"
and
> imagine including every header into every .cpp file :)
>
Yeah, I remember that. I was actually asking about the new improvements you
mentioned in the internal thread. That's what confused me. I thought you
were referring to those.

Thanks for the clarification.

Diego.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141017/f20167f8/attachment.html>