Óscar Fuentes
2011-Nov-01 23:16 UTC
[LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes
Nico Weber <thakis at chromium.org> writes:> On Tue, Nov 1, 2011 at 3:09 PM, David A. Greene <greened at obbligato.org> wrote: >> Óscar Fuentes <ofv at wanadoo.es> writes: >> >>> Okay, we can get rid of recursive make. However, as pointed out >>> elsewhere, removing recursive make will not make a difference on the >>> LLVM build. What David Greene says probably is related to difference >>> sizes among his compiler and LLVM. > > I doubt that's true: Chromium's empty build is faster than llvm's > (only tried the make build, not the cmake build), and chromium's code > base is considerably larger.What David was saying is that Cray's compiler *non-empty* build ends sooner than the LLVM *empty* build completes. Well, if only a small translation unit needs to be compiled into a small executable (no long compiler/linker waits) it may be possible but, again, it only speaks about the relative sizes of LLVM/Cray's compiler. WRT your Chromium example, the crux is not so much about what is faster, but about what is considered slow. Chromium's empty build being faster than LLVM is of no significance if LLVM's empty build can't be considered annoyingly slow. And I wouldn't say that 1.3 seconds on a rather ordinary desktop is annoyingly slow.
David A. Greene
2011-Nov-01 23:27 UTC
[LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes
Óscar Fuentes <ofv at wanadoo.es> writes:> Nico Weber <thakis at chromium.org> writes: > >> On Tue, Nov 1, 2011 at 3:09 PM, David A. Greene <greened at obbligato.org> wrote: >>> Óscar Fuentes <ofv at wanadoo.es> writes: >>> >>>> Okay, we can get rid of recursive make. However, as pointed out >>>> elsewhere, removing recursive make will not make a difference on the >>>> LLVM build. What David Greene says probably is related to difference >>>> sizes among his compiler and LLVM. >> >> I doubt that's true: Chromium's empty build is faster than llvm's >> (only tried the make build, not the cmake build), and chromium's code >> base is considerably larger. > > What David was saying is that Cray's compiler *non-empty* build ends > sooner than the LLVM *empty* build completes. Well, if only a small > translation unit needs to be compiled into a small executable (no long > compiler/linker waits) it may be possible but, again, it only speaks > about the relative sizes of LLVM/Cray's compiler.Ok, here are some hard numbers for empty builds: LLVM empty build: /usr/bin/time make -j16 4.32user 2.47system 0:03.21elapsed 211%CPU (0avgtext+0avgdata 13376maxresident)k 0inputs+0outputs (0major+671804minor)pagefaults 0swaps Cray empty build: /usr/bin/time make dynamic-developer -j16 2.88user 1.06system 0:04.94elapsed 79%CPU (0avgtext+0avgdata 64208maxresident)k 0inputs+0outputs (0major+184605minor)pagefaults 0swaps The Cray empty build includes some really horrendous shell script trickery known as "modules." You can read the gory details here: http://modules.sourceforge.net/ It's possibly the dumbest thing ever invented. But that's what I've got to work with. The point is, the Cray build includes some major shell overhead that the LLVM build doesn't have to deal with. And the Cray build has more targets to build. And the Cray build has more dependencies to examine. And the Cray build still blows the socks off the LLVM build. Here is actual data comparing an empty LLVM build done recursively (the LLVM build) and non-recursively (the Cray build). See this? 0inputs+0outputs (0major+671804minor)pagefaults 0swaps vs. this? 0inputs+0outputs (0major+184605minor)pagefaults 0swaps That's I/O. -Dave
Joerg Sonnenberger
2011-Nov-01 23:53 UTC
[LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes
On Tue, Nov 01, 2011 at 06:27:35PM -0500, David A. Greene wrote:> Here is actual data comparing an empty LLVM build done recursively (the > LLVM build) and non-recursively (the Cray build). > > See this? > > 0inputs+0outputs (0major+671804minor)pagefaults 0swaps > > vs. this? > > 0inputs+0outputs (0major+184605minor)pagefaults 0swaps > > That's I/O.Sorry to disappoint you, but no, this isn't I/O. You have just shown that a LLVM build touches more mmap'd pages. Joerg
Óscar Fuentes
2011-Nov-02 00:12 UTC
[LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes
greened at obbligato.org (David A. Greene) writes:> Ok, here are some hard numbers for empty builds: > > LLVM empty build: > /usr/bin/time make -j16 > 4.32user 2.47system 0:03.21elapsed 211%CPU (0avgtext+0avgdata 13376maxresident)k > 0inputs+0outputs (0major+671804minor)pagefaults 0swapsSo your 16-way machine takes 2.5 times more than my 4-way cheap desktop...> Cray empty build: > /usr/bin/time make dynamic-developer -j16 > 2.88user 1.06system 0:04.94elapsed 79%CPU (0avgtext+0avgdata 64208maxresident)k > 0inputs+0outputs (0major+184605minor)pagefaults 0swaps > > The Cray empty build includes some really horrendous shell script > trickery known as "modules." You can read the gory details here: > > http://modules.sourceforge.net/ > > It's possibly the dumbest thing ever invented. But that's what I've got > to work with. > > The point is, the Cray build includes some major shell overhead that the > LLVM build doesn't have to deal with. And the Cray build has more > targets to build. And the Cray build has more dependencies to examine. > And the Cray build still blows the socks off the LLVM build.... and you hypothesize that the shell work adds substantial time to the Cray build, thus making even more dramatic the difference on the LLVM-Cray build systems.> Here is actual data comparing an empty LLVM build done recursively (the > LLVM build) and non-recursively (the Cray build). > > See this? > > 0inputs+0outputs (0major+671804minor)pagefaults 0swaps > > vs. this? > > 0inputs+0outputs (0major+184605minor)pagefaults 0swaps > > That's I/O.Let me guess: you are starting from a cold cache. Other than a difference caused by the OS, that's the only thing that could explain it. The format used by your `time' is not familiar to me, so excuse me if I make a question related to your measures: LLVM: 4.32user 2.47system 0:03.21elapsed 211%CPU Cray: 2.88user 1.06system 0:04.94elapsed 79%CPU Looks like the Cray build is using less CPU but takes 1.7 seconds more than LLVM of wall clock time. But then you note that the LLVM build uses a lot more I/O. How can this be? It looks as if the LLVM build were much more parallel. Is this explained because the shell scripts you mention above introduce serialization points? And the decisive question: is 3.2 seconds considered an annoying amount of time for you deciding to support a substantial change on the LLVM build, but 4.94 seconds is not annoying enough to get rid of those shell scripts? :-)