On Mon, Sep 26, 2016 at 8:08 AM, Carsten Mattner <carstenmattner at gmail.com> wrote:> On Mon, Sep 26, 2016 at 4:25 PM, Teresa Johnson <tejohnson at google.com> > wrote: > > No worries, thanks for the update. Teresa > > 2048 wasn't enough. Bumped to 4096. Only 1300 ninja targets left. > > Once I've been successful with this, I might try building a ThinLTO Firefox > or maybe Chromium,I have built both of those successfully with ThinLTO, so they should work.> although browsers usually have PGO profiles one > can run for feeding back real-world exercise. Not sure if PGO can be > combined with LTO, or if it makes sense at all... >Yes, absolutely! ThinLTO and PGO are largely orthogonal, but PGO will help enable more indirect call promotion with ThinLTO, and there is also a patch out for review (D24638) to use FDO to guide ThinLTO function importing heuristics.> What would be more fun is combining ThinLTO for C code with MLton > for SML code when building Ur/Web. >Interesting. I am not at all familiar with SML code/building, but theoretically switching the C code to use -flto=thin shouldn't affect the interaction between those two compiles. Presumably one or both are compiled into a library and linked in to create a final binary? Teresa> > > On Mon, Sep 26, 2016, 7:16 AM Carsten Mattner <carstenmattner at gmail.com> > > wrote: > >> > >> On Mon, Sep 26, 2016 at 3:52 PM, Teresa Johnson <tejohnson at google.com> > >> wrote: > >> > > >> > ThinLTO needs to create as many temporary files as there are input > >> > modules to the link. From your 'ls' below, it doesn't look like there > is > >> > an incredibly huge amount already created, can you check your limits? > >> > What is the output of: > >> > > >> > $ ulimit -n > >> > > >> > and > >> > > >> > $ sysctl fs.file-max > >> > >> Sorry for the noise, forgot to verify because I had assumed it was > >> still set to 4096 from a similar change a couple days ago. > >> Alas, it's just 1024 in this session. Given the # of object files, > >> 2048 looks more than enough. Going to re-run with that. >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160926/3e348fb9/attachment.html>
$ /usr/bin/time ninja -j4 ... [1223/1315] Linking CXX executable bin/lldb-3.9.1 FAILED: bin/lldb-3.9.1 [long list of object files as part of clang++ invocation command] /usr/bin/ld: bin/lldb-3.9.1: hidden symbol `__morestack' in /usr/lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o) is referenced by DSO /usr/bin/ld: final link failed: Bad value ... 27428.01user 218.31system 1:59:40elapsed 385%CPU (0avgtext+0avgdata 980356maxresident)k 931232inputs+4172128outputs (1390major+36787899minor)pagefaults 0swaps Failed to link lldb, while less then 100 ninja targets were left. Same if I retry with ninja -j1. Any suggestions?
On Mon, Sep 26, 2016 at 5:16 PM, Teresa Johnson <tejohnson at google.com> wrote:> > > > On Mon, Sep 26, 2016 at 8:08 AM, Carsten Mattner <carstenmattner at gmail.com> wrote: >> >> On Mon, Sep 26, 2016 at 4:25 PM, Teresa Johnson <tejohnson at google.com> wrote: >> > No worries, thanks for the update. Teresa >> >> 2048 wasn't enough. Bumped to 4096. Only 1300 ninja targets left. >> >> Once I've been successful with this, I might try building a ThinLTO Firefox >> or maybe Chromium, > > > I have built both of those successfully with ThinLTO, so they should work.Sounds good.>> although browsers usually have PGO profiles one >> can run for feeding back real-world exercise. Not sure if PGO can be >> combined with LTO, or if it makes sense at all... > > > Yes, absolutely! ThinLTO and PGO are largely orthogonal, but PGO will help enable > more indirect call promotion with ThinLTO, and there is also a patch out for review (D24638) > to use FDO to guide ThinLTO function importing heuristics.Also sounds good.>> What would be more fun is combining ThinLTO for C code with MLton >> for SML code when building Ur/Web. > > > Interesting. I am not at all familiar with SML code/building, but theoretically switching the C code to > use -flto=thin shouldn't affect the interaction between those two compiles. Presumably one or both > are compiled into a library and linked in to create a final binary?Ur/Web consists of C and SML code, and I'd have to take a deep dive before I could say how it's linked, but basically yes,the finaly piece is a combination of object code from C and SML sources. I thought given MLton's whole program optimizer, ThinLTO might be natural extension of the build process. Just because :).
On Mon, Sep 26, 2016 at 10:07 AM, Carsten Mattner <carstenmattner at gmail.com> wrote:> $ /usr/bin/time ninja -j4 > ... > [1223/1315] Linking CXX executable bin/lldb-3.9.1 > FAILED: bin/lldb-3.9.1 > [long list of object files as part of clang++ invocation command] > /usr/bin/ld: bin/lldb-3.9.1: hidden symbol `__morestack' in > /usr/lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o) > is referenced by DSO > /usr/bin/ld: final link failed: Bad value > ... > 27428.01user 218.31system 1:59:40elapsed 385%CPU (0avgtext+0avgdata > 980356maxresident)k > 931232inputs+4172128outputs (1390major+36787899minor)pagefaults 0swaps > > Failed to link lldb, while less then 100 ninja targets were left. > > Same if I retry with ninja -j1. > > Any suggestions? >I did some poking around on the web and found that this exact issue in the lldb link cropped up due to a patch: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20141229/250420.html It was fixed by making the symbol weak: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20141229/250420.html I'm wondering if ThinLTO is mucking with the symbol linkage somehow and effectively undoing that weak resolution. However, typically we change the linkage type only for definitions, and in this case the def is in libgcc.a, not in LLVM. So I am not completely sure what is going on. I'll either need to get a reproducer from you and/or try to repro it myself. I usually don't check out and build lldb, so perhaps I will start with that. As a workaround you could try -DBUILD_SHARED_LIBS=OFF. Teresa -- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160926/584a1265/attachment.html>