No worries, thanks for the update. Teresa On Mon, Sep 26, 2016, 7:16 AM Carsten Mattner <carstenmattner at gmail.com> wrote:> On Mon, Sep 26, 2016 at 3:52 PM, Teresa Johnson <tejohnson at google.com> > wrote: > > > > ThinLTO needs to create as many temporary files as there are input > > modules to the link. From your 'ls' below, it doesn't look like there is > > an incredibly huge amount already created, can you check your limits? > > What is the output of: > > > > $ ulimit -n > > > > and > > > > $ sysctl fs.file-max > > Sorry for the noise, forgot to verify because I had assumed it was > still set to 4096 from a similar change a couple days ago. > Alas, it's just 1024 in this session. Given the # of object files, > 2048 looks more than enough. Going to re-run with that. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160926/e8e2732e/attachment.html>
On Mon, Sep 26, 2016 at 4:25 PM, Teresa Johnson <tejohnson at google.com> wrote:> No worries, thanks for the update. Teresa2048 wasn't enough. Bumped to 4096. Only 1300 ninja targets left. Once I've been successful with this, I might try building a ThinLTO Firefox or maybe Chromium, although browsers usually have PGO profiles one can run for feeding back real-world exercise. Not sure if PGO can be combined with LTO, or if it makes sense at all... What would be more fun is combining ThinLTO for C code with MLton for SML code when building Ur/Web.> On Mon, Sep 26, 2016, 7:16 AM Carsten Mattner <carstenmattner at gmail.com> > wrote: >> >> On Mon, Sep 26, 2016 at 3:52 PM, Teresa Johnson <tejohnson at google.com> >> wrote: >> > >> > ThinLTO needs to create as many temporary files as there are input >> > modules to the link. From your 'ls' below, it doesn't look like there is >> > an incredibly huge amount already created, can you check your limits? >> > What is the output of: >> > >> > $ ulimit -n >> > >> > and >> > >> > $ sysctl fs.file-max >> >> Sorry for the noise, forgot to verify because I had assumed it was >> still set to 4096 from a similar change a couple days ago. >> Alas, it's just 1024 in this session. Given the # of object files, >> 2048 looks more than enough. Going to re-run with that.
On Mon, Sep 26, 2016 at 8:08 AM, Carsten Mattner <carstenmattner at gmail.com> wrote:> On Mon, Sep 26, 2016 at 4:25 PM, Teresa Johnson <tejohnson at google.com> > wrote: > > No worries, thanks for the update. Teresa > > 2048 wasn't enough. Bumped to 4096. Only 1300 ninja targets left. > > Once I've been successful with this, I might try building a ThinLTO Firefox > or maybe Chromium,I have built both of those successfully with ThinLTO, so they should work.> although browsers usually have PGO profiles one > can run for feeding back real-world exercise. Not sure if PGO can be > combined with LTO, or if it makes sense at all... >Yes, absolutely! ThinLTO and PGO are largely orthogonal, but PGO will help enable more indirect call promotion with ThinLTO, and there is also a patch out for review (D24638) to use FDO to guide ThinLTO function importing heuristics.> What would be more fun is combining ThinLTO for C code with MLton > for SML code when building Ur/Web. >Interesting. I am not at all familiar with SML code/building, but theoretically switching the C code to use -flto=thin shouldn't affect the interaction between those two compiles. Presumably one or both are compiled into a library and linked in to create a final binary? Teresa> > > On Mon, Sep 26, 2016, 7:16 AM Carsten Mattner <carstenmattner at gmail.com> > > wrote: > >> > >> On Mon, Sep 26, 2016 at 3:52 PM, Teresa Johnson <tejohnson at google.com> > >> wrote: > >> > > >> > ThinLTO needs to create as many temporary files as there are input > >> > modules to the link. From your 'ls' below, it doesn't look like there > is > >> > an incredibly huge amount already created, can you check your limits? > >> > What is the output of: > >> > > >> > $ ulimit -n > >> > > >> > and > >> > > >> > $ sysctl fs.file-max > >> > >> Sorry for the noise, forgot to verify because I had assumed it was > >> still set to 4096 from a similar change a couple days ago. > >> Alas, it's just 1024 in this session. Given the # of object files, > >> 2048 looks more than enough. Going to re-run with that. >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160926/3e348fb9/attachment.html>
> On Sep 27, 2016, at 2:18 AM, Carsten Mattner <carstenmattner at gmail.com> wrote: > >> On Mon, Sep 26, 2016 at 11:02 PM, Teresa Johnson <tejohnson at google.com> wrote: >> I'll either need to get a reproducer from you and/or try to repro it myself. > > Assuming I run into this with BUILD_SHARED_LIBS=OFF, which seems unlikely, > what kind of reproducer data would you need from me? > >> I usually don't check out and build lldb, so perhaps I will start with that. > > I must say I'm surprised a ThinLTO build of all of LLVM svn isn't part of > the periodic builders.I have one, it just does include LLDB. Mehdi> >> As a workaround you could try -DBUILD_SHARED_LIBS=OFF. > > Thanks, will do, but probably not before Wednesday because it looks like > this requires a full rebuild, if my interpretation of the cmake warning/error > is correct.
On Tue, Sep 27, 2016 at 6:53 AM, Mehdi Amini <mehdi.amini at apple.com> wrote:> > > > On Sep 27, 2016, at 2:18 AM, Carsten Mattner <carstenmattner at gmail.com> > wrote: > > > >> On Mon, Sep 26, 2016 at 11:02 PM, Teresa Johnson <tejohnson at google.com> > wrote: > >> I'll either need to get a reproducer from you and/or try to repro it > myself. > > > > Assuming I run into this with BUILD_SHARED_LIBS=OFF, which seems > unlikely, > > what kind of reproducer data would you need from me? >Let me see if I can reproduce it first. Probably a zip of all the inputs to the link and the link command line. Another option would be to save the temp files, but it would require some work to find the one producing the error and I might need all the inputs anyway...> > > >> I usually don't check out and build lldb, so perhaps I will start with > that. > > > > I must say I'm surprised a ThinLTO build of all of LLVM svn isn't part of > > the periodic builders. > > > I have one, it just does include LLDB. >Great, I was going to ask how to get one set up. Is it an official build bot? And I assume you meant it just does "not" include LLDB? Working getting a shared build of LLDB right now... Teresa> Mehdi > > > >> As a workaround you could try -DBUILD_SHARED_LIBS=OFF. > > > > Thanks, will do, but probably not before Wednesday because it looks like > > this requires a full rebuild, if my interpretation of the cmake > warning/error > > is correct. >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160927/8deae87e/attachment.html>