search for: pr17891

...com> wrote: > > Author: dexonsmith > Date: Thu Oct 2 16:56:57 2014 > New Revision: 218914 > > URL: http://llvm.org/viewvc/llvm-project?rev=218914&view=rev > Log: > DI: Fold constant arguments into a single MDString > > This patch addresses the first stage of PR17891 by folding constant > arguments together into a single MDString. Integers are stringified and > a `\0` character is used as a separator. > > Part of PR17891. > > Note: I've attached my testcases upgrade scripts to the PR. If I've > just broken your out-of-tree testc...

...came into this expecting to work with Eric on parallelizing the backend, but consistently found that callback-based RAUW traffic for metadata took almost as much CPU. Since debug info IR is at the heart of the RAUW bottleneck, I looked into its memory layout (it's a hog). I started working on PR17891 because, besides improving the memory usage, the plan promised to greatly reduce the number of nodes (indirectly reducing RAUW traffic). In the context of `llvm-lto`, "stage 1" knocked memory usage down from ~5GB to ~3GB -- but didn't reduce the number of nodes. Before starting stag...

...#39;t >> showing up in a profile, I may skip this. > On 2014 Oct 18, at 10:27, Sean Silva <chisophugis at gmail.com> wrote: > >> Since debug info IR is at the heart of the RAUW bottleneck, I looked >> into its memory layout (it's a hog). I started working on PR17891 >> because, besides improving the memory usage, the plan promised to >> greatly reduce the number of nodes (indirectly reducing RAUW traffic). >> >> In the context of `llvm-lto`, "stage 1" knocked memory usage down from >> ~5GB to ~3GB -- but didn't redu...

...pe uniquing: 3.4GB MDNodes after parsing all bc files, 7GB MDNodes after linking all bc files 4.6GB DIEs 4G MCContext --> The memory usage is still too big. So how to reduce the memory footprint at MDNode level: 1> Combine integers into MDString and further combining MDStrings (see PR17891) A partial implementation on the important debug info nodes can reduce the MDNodes from 7GB to 5.7GB 2> Release MDNodes that are only used by source modules (I will send out a proposal) An estimation based on partial implementation: this will reduce MDNodes from 5.7GB to 3.9GB...

...> --> The memory usage is still too big. > Do we have an idea of what size is "small enough"? It would be useful to have a goal. > So how to reduce the memory footprint at MDNode level: > 1> Combine integers into MDString and further combining MDStrings (see > PR17891) > A partial implementation on the important debug info nodes can > reduce the MDNodes from 7GB to 5.7GB > I think this'll be an interesting, and potentially valuable, change even in non-LTO cases, but not necessarily where I would start just now. > 2> Release MDNodes...

...wrote: >> >> In r219010, I merged integer and string fields into a single header >> field. By reducing the number of metadata operands used in debug info, >> this saved 2.2GB on an `llvm-lto` bootstrap. I've done some profiling >> of DW_TAGs to see what parts of PR17891 and PR17892 to tackle next, and >> I've concluded that they will be insufficient. >> >> Instead, I'd like to implement a more aggressive plan, which as a >> side-effect cleans up the much "loved" debug info IR assembly syntax. >> >> At a high-le...

In r219010, I merged integer and string fields into a single header field. By reducing the number of metadata operands used in debug info, this saved 2.2GB on an `llvm-lto` bootstrap. I've done some profiling of DW_TAGs to see what parts of PR17891 and PR17892 to tackle next, and I've concluded that they will be insufficient. Instead, I'd like to implement a more aggressive plan, which as a side-effect cleans up the much "loved" debug info IR assembly syntax. At a high-level, the idea is to create distinct subclasses of `V...

...too big. >> > > Do we have an idea of what size is "small enough"? It would be useful to > have a goal. > > >> So how to reduce the memory footprint at MDNode level: >> 1> Combine integers into MDString and further combining MDStrings (see >> PR17891) >> A partial implementation on the important debug info nodes can >> reduce the MDNodes from 7GB to 5.7GB >> > > I think this'll be an interesting, and potentially valuable, change even > in non-LTO cases, but not necessarily where I would start just now. >...

...; >> In r219010, I merged integer and string fields into a single header > >> field. By reducing the number of metadata operands used in debug info, > >> this saved 2.2GB on an `llvm-lto` bootstrap. I've done some profiling > >> of DW_TAGs to see what parts of PR17891 and PR17892 to tackle next, and > >> I've concluded that they will be insufficient. > >> > >> Instead, I'd like to implement a more aggressive plan, which as a > >> side-effect cleans up the much "loved" debug info IR assembly syntax. > >&...

...onsmith at apple.com> wrote: > In r219010, I merged integer and string fields into a single header > field. By reducing the number of metadata operands used in debug info, > this saved 2.2GB on an `llvm-lto` bootstrap. I've done some profiling > of DW_TAGs to see what parts of PR17891 and PR17892 to tackle next, and > I've concluded that they will be insufficient. > > Instead, I'd like to implement a more aggressive plan, which as a > side-effect cleans up the much "loved" debug info IR assembly syntax. > > At a high-level, the idea is to crea...

> On 2014 Oct 16, at 22:09, Sean Silva <chisophugis at gmail.com> wrote: > > Dig into this first! This isn't the right forum for digging into ld64. > In the OP you are talking about essentially a pure "optimization" (in the programmer-wisdom "beware of it" sense), to "save" 2GB of peak memory. But from your analysis it's not clear that