One more update. The Makefile.am for analyze was wrong. It wasn't linking in the some of the passes. The new size is 56951088 which is in line with the other executables. Also, I have now completed a run of projects/llvm-test/MultiSource with the tools generated by automake. The only errors were for: TEST (llc) 'sgefa' FAILED! TEST (jit) 'sgefa' FAILED! TEST (jit) 'make_dparser' FAILED! TEST (llc) 'kc' FAILED! TEST (jit) 'anagram' FAILED! TEST (llc) 'mason' FAILED! TEST (cbe) 'mason' FAILED! TEST (jit) 'mason' FAILED! TEST (jit) 'pcompress2' FAILED! TEST (jit) 'make' FAILED! TEST (cbe) 'timberwolfmc' FAILED! TEST (jit) 'agrep' FAILED! Which isn't far off what the nightly test produces these days. Reid. On Sun, 2004-10-17 at 20:05, Reid Spencer wrote:> After puzzling about the size of the executables and the build times, > I discovered (thanks Chris!) that I had compiled everything without > debug symbols in the automake version. So, here's some revision from the > first version of this email. > > The build times didn't change much (I guess I/O is cheap on my machine). > The new "Build With Automake" times are 20m28.672s (elapsed), 18m1.900s > (user) and 1m38.540s (system). > > The real change is in the size of the executables. Thew new values, > while still smaller are much more reasonable. Previously the automake > build and existing build were using different flags to the compiler. The > results below are with the same flags (I double checked). > > Automake Existing Pct Name > 16903982 46046545 37% analyze (bug in existing makefiles) > 73084123 77679274 94% bugpoint > 17638401 19137945 92% extract > 37945217 47578060 80% gccas > 31870129 34163210 93% gccld > 56967280 60263187 95% llc > 48570878 52162647 93% lli > 15040029 16435732 92% llvm-as > 50580919 54185542 93% llvm-db > 14306895 15667554 91% llvm-dis > 69413397 73995210 94% opt > > Sorry for the confusion. > > Reid. > > On Sat, 2004-10-16 at 23:25, Reid Spencer wrote: > > Folks, > > > > I have completed the addition of automake makefiles to LLVM. All > > libraries, tools, and runtime libs build now with automake. Note that > > there are still many missing things in the automake support. Right now > > it just builds the basic software. > > > > However, before I invest more time in it, I thought some comparison > > would help us make some decisions about whether or not to proceed with > > automake for the LLVM standard. There are costs and benefits on both > > sides. > > > > BUILD/CONFIGURE TIMES > > ================================================================> > Item Elapsed User System > > ----------------------------- ---------- ---------- ---------- > > Configure With Automake 0m48.835s 0m22.700s 0m21.890s > > Configure With Current System 0m37.939s 0m18.230s 0m16.980s > > Build With Automake 20m29.057s 18m30.230s 1m28.080s > > Build With Current System 30m56.017s 25m56.750s 3m15.790s > > > > This was recorded on a 2x2.4GHz Xeon RH9 Linux machine (same one used > > for the optimized nightly test). The build was run with "gmake -j 1" in > > both cases. The time includes the total time taken to build utils, lib, > > tools, and runtime. > > > > Note that while the automake configure takes longer, automake builds the > > software in about 1/3 less time than our existing system. A "gmake -j 3" > > builds from scratch on my machine in under 10 minutes. Since building is > > done much more frequently than configuring, this is a development win. > > > > EXECUTABLE SIZES > > ===============> > > > Below is a list of the sizes of some of the larger/important executables > > that LLVM builds. The bytesize of the Automake built version and the > > Existing version are shown as well as a percentage. In every case the > > automake executable is smaller, by an order of magnitude. > > > > AutoMake Existing Pct Program > > -------- -------- ---- ------- > > 2084426 46046545 5% analyze > > 6118914 77679274 8% bugpoint* > > 2038252 19137945 11% extract* > > 3817030 47578060 8% gccas* > > 3244568 34163210 9% gccld* > > 5713818 60263187 9% llc* > > 6446641 52162647 12% lli* > > 1892254 16435732 12% llvm-as* > > 6630877 54185542 12% llvm-db* > > 1811843 15667554 12% llvm-dis* > > 5752934 73995210 8% opt > > 556978 5153127 11% tblgen > > > > I tried to explain this with two things: -On options. The existing > > makefiles build without using -O. I've configure automake to use -O1 on > > its compilations. That could still be the difference but I don't see how > > it explains an order of magnitude. Another possibility was that I just > > wasn't linking in all the static (pre-linked) object libraries in some > > of the executables. But that doesn't explain llvm-as or llvm-db and I > > have very meticulously made sure that what gets linked is the same as > > the existing system (if I deviated, usually it didn't link). All the > > programs run and produces the same --help out. That's significant > > because if certain libraries were not being linked in, their command > > line options wouldn't appear in the --help output. > > > > The only thing I can think of is that the existing makefiles are doing > > something weird that causes the bloat (probably in the debug info). > > > > automake PROS/CONS > > =================> > + builds faster > > + builds smaller executables > > + we don't have to maintain it > > + new features/platforms are an upgrade away > > + standard makefile system known by lots of developers > > + GNU Makefile Standards compliance > > + rules are tried and tested and work well on lots of platforms > > + supports automatically creating a distribution (dist target) > > + supports automatically testing a distribution (distcheck target) > > + fast/standard install/uninstall targets > > + handles install/uninstall of scripts, headers, data, etc. > > + handles installcheck (checking programs after installation) > > + supports automated testing via dejagnu > > + completely groks libtool for building shared libs on lots of platforms > > + autoconf flags can determine nature of the build > > + supports C, C++, Fortran, Java, Obj-C, Lex, Yacc, Python > > + knows how to build texinfo & man page documentation > > + perfect dependency tracking (each compile updates the info) > > + automatically handles .exe (or other) extensions for executables > > wherever its needed > > + automatically handles C/C++ tags/etags > > + supports building different targets with different compile options > > + can still be extended in any way GNU make can be > > - have to specify each source input file (currently searching > > for a workaround) > > - completely dependent on gmake > > - requires Perl (automake written in Perl) > > > > existing PROS/CONS > > =================> > + its done and working > > + we have full control over anything > > + somewhat similar to the BSD makefile system > > - completely dependent on gmake > > - we must maintain it ourselves > > - doesn't handle automatic distribution generation > > - installs slowly > > - testing support is adhoc/inconsistent > > > > CURRENT THINGS TO FIX/FINISH IN AUTOMAKE RULES > > =============================================> > > > 1. The default/only mode of output is VERBOSE. > > 2. Don't use more Makefile.am files than is necessary (e.g. SparcV9 > > should just have one, not 5) > > 3. Find a way to make it handle bytecode output better > > 4. Find a way to not maintain list of sources if the library > > developer so chooses (i.e. assume all sources found) > > 5. Add support in test, examples, projects directories > > > > Reid > > > > ______________________________________________________________________ > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20041017/89257b78/attachment.sig>
I'm re-thinking my penchant for automake. automake is great for many standard applications that just need to get portable makefiles up and running quickly. However, it turns out that LLVM is "different enough" from a standard application that automake might not be the best choice. Here's some of the problems I've run into: 1. There's no way to tell automake to build bytecode. Without a lot of customization of automake itself, it just can't grok the fact that there might be multiple ways to compile a c/c++ program producing different results and requiring different tools. 2. The entire llvm-test project would have to either be completely rethought or stay the way it is, it just can't be easily automake'd because it doesn't follow the automake pattern. 3. The llvm/test directory would require significant rework to get it to automake correctly (via dejagnu). 4. There's no way to avoid listing all the sources for every library (I've tried several alternatives with no good results). 5. There's no way to install bytecode libraries somewhere separately from other libraries. 6. Creating a distribution tarball would require additional Makefile.am files to be inserted in all the directories under llvm/include, the traversal of which would cause additional overhead for every non-dist target. That's 17 gratuitous "make" processes per build. There doesn't seem to be a way around this. 7. I can't get automake to stop doing a double configure. That is, when you configure all the Makefiles are updated. You then go do a build and it thinks it needs to reconfigure again so it does it twice. This is just a waste of time (40 seconds on my machine). 8. automake's notion of buiding a library is very fixed. It basically supports two things: libtool built shared libraries (linking) and ar built libraries that are also run through ranlib. While its possible to override AR and LINK, there isn't a way to override RANLIB on a per-target basis so you can't build both a regular library (requiring RANLIB=ranlib) and a bytecode library (requiring something like RANLIB=true) in the same directory. We have in LLVM at least 4 ways to build libraries: regular .a, pre-linked .o (combination of .o files), shared-library, and bytecode-library. While I figured out a hack to do pre-linked .o, it basically uses GNU Make, not automake to make it work and therefore breaks automake's "make" portability. 9. There's probably a bunch of things I haven't run into yet. SOOOOOOO .... Instead of spending a bunch more time on trying to get automake to work, I suggest we just fix the current makefile system to do what automake can do. Specifically we need to: 1. Get dependency generation in a single pass like automake. This will give us about a 30% speed up on builds .. and one of the main reasons for moving to automake is taken away. Estimate: 1-2 hours. 2. Get targets for building distributions and checking them. For this we can look at what automake generates and just mimic it. Estimate: 1 day. 3. Get targets for install and uninstall working correctly and make proper use of the install command (instead of relying on libtool) so that installation goes faster. Estimate: 1 day. 4. Get "make check" to work by allowing any directory to have a set of programs to be run that "check" that directory. These could be shell scripts or whatever. Estimate: 1-2 days. 5. Integrate dejagnu/expect/other into "make check" so that our make system can run tests that parse program output to categorize the result as PASS/FAIL/XFAIL/XPASS. This would eliminate our dependence on qmtest and make all input/output for the tests be plain text that is easily editable (instead of in a database). Estimate: 1-2 weeks. I am, of course, soliciting feedback on this whole idea. Reid. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20041020/c554190e/attachment.sig>
On Wed, Oct 20, 2004 at 01:01:33AM -0700, Reid Spencer wrote:> Instead of spending a bunch more time on trying to get automake to work, > I suggest we just fix the current makefile system to do what automake > can do. Specifically we need to:[snip]> I am, of course, soliciting feedback on this whole idea.I would agree that, given the differences, it is better to improve the current system to do what automake can do than to switch to automake and teach automake how to do things that our current build system already does (which in some cases, as you mention, may not be reasonable). I am not sufficiently familiar with dependency generation, et al, to comment on it in detail, but I would *love* a "make check" facility with results listed in plain-text files rather than a database that required running qmtest, logging into it via a web browser, and updating the binary DB that way. And if there's a "make dist" and/or "make rpm" target(s), so much the better. -- Misha Brukman :: http://misha.brukman.net :: http://llvm.cs.uiuc.edu
On Wednesday 20 October 2004 12:01, Reid Spencer wrote:> I'm re-thinking my penchant for automake. automake is great for many > standard applications that just need to get portable makefiles up and > running quickly. However, it turns out that LLVM is "different enough" > from a standard application that automake might not be the best choice.I might just here to suggest that Boost.Build (http://boost.org/boost-build2) might be good.> Here's some of the problems I've run into: > > 1. There's no way to tell automake to build bytecode. Without a lot of > customization of automake itself, it just can't grok the fact that there > might be multiple ways to compile a c/c++ program producing different > results and requiring different tools.Boost.Build supports this very good. In particular, on my project on work I can run: bjam toolset=gcc bjam toolset=nm bjam toolset=nmm and the first will do a regular build, the third will compile for some embedded processor, and the third will pass the sources via some annotation tool and produce an annotated x86 binary. Besides, "toolset=gcc" can become "toolset=msvc" on windows, and it will have high chances of working.> 2. The entire llvm-test project would have to either be completely > rethought or stay the way it is, it just can't be easily automake'd > because it doesn't follow the automake pattern. > > 3. The llvm/test directory would require significant rework to get it to > automake correctly (via dejagnu).No comments on the two points above yet.> 4. There's no way to avoid listing all the sources for every library > (I've tried several alternatives with no good results).Easy, I've already have a half-working setup (only the targets I use are converted), and, for example, lib/Analysis/DataStructure/Jamfile has nothing but: llvm-lib datastructure ;> 5. There's no way to install bytecode libraries somewhere separately > from other libraries.Should be possible.> 6. Creating a distribution tarball would require additional Makefile.am > files to be inserted in all the directories under llvm/include, the > traversal of which would cause additional overhead for every non-dist > target. That's 17 gratuitous "make" processes per build. There doesn't > seem to be a way around this.Why distribution tarball has anything to do with build system?> 7. I can't get automake to stop doing a double configure. That is, when > you configure all the Makefiles are updated. You then go do a build and > it thinks it needs to reconfigure again so it does it twice. This is > just a waste of time (40 seconds on my machine).Ehm.... I'd say that configure should be orthogonal to building. No?> 8. automake's notion of buiding a library is very fixed. It basically > supports two things: libtool built shared libraries (linking) and ar > built libraries that are also run through ranlib. While its possible to > override AR and LINK, there isn't a way to override RANLIB on a > per-target basis so you can't build both a regular library (requiring > RANLIB=ranlib) and a bytecode library (requiring something like > RANLIB=true) in the same directory. We have in LLVM at least 4 ways to > build libraries: regular .a, pre-linked .o (combination of .o files), > shared-library, and bytecode-library. While I figured out a hack to do > pre-linked .o, it basically uses GNU Make, not automake to make it work > and therefore breaks automake's "make" portability.I think given Boost.Build's "properties" (like "toolset" above), this should be doable.> 1. Get dependency generation in a single pass like automake. This will > give us about a 30% speed up on builds .. and one of the main reasons > for moving to automake is taken away. Estimate: 1-2 hours.Well, Boost.Build is single pass already. Need to compare performance on LLVM, though. Basically, I can try to finish my attempt to add Boost.Build files to LLVM. But I wonder if that makes sense? Is it already decided to just improve the current system? Is anything with "Boost" in name will be rejected right away? - Volodya