Folks,
I have completed the addition of automake makefiles to LLVM. All
libraries, tools, and runtime libs build now with automake. Note that
there are still many missing things in the automake support. Right now
it just builds the basic software.
However, before I invest more time in it, I thought some comparison
would help us make some decisions about whether or not to proceed with
automake for the LLVM standard. There are costs and benefits on both
sides.
BUILD/CONFIGURE TIMES
================================================================Item
Elapsed User System
----------------------------- ---------- ---------- ----------
Configure With Automake 0m48.835s 0m22.700s 0m21.890s
Configure With Current System 0m37.939s 0m18.230s 0m16.980s
Build With Automake 20m29.057s 18m30.230s 1m28.080s
Build With Current System 30m56.017s 25m56.750s 3m15.790s
This was recorded on a 2x2.4GHz Xeon RH9 Linux machine (same one used
for the optimized nightly test). The build was run with "gmake -j 1"
in
both cases. The time includes the total time taken to build utils, lib,
tools, and runtime.
Note that while the automake configure takes longer, automake builds the
software in about 1/3 less time than our existing system. A "gmake -j
3"
builds from scratch on my machine in under 10 minutes. Since building is
done much more frequently than configuring, this is a development win.
EXECUTABLE SIZES
===============
Below is a list of the sizes of some of the larger/important executables
that LLVM builds. The bytesize of the Automake built version and the
Existing version are shown as well as a percentage. In every case the
automake executable is smaller, by an order of magnitude.
AutoMake Existing Pct Program
-------- -------- ---- -------
2084426 46046545 5% analyze
6118914 77679274 8% bugpoint*
2038252 19137945 11% extract*
3817030 47578060 8% gccas*
3244568 34163210 9% gccld*
5713818 60263187 9% llc*
6446641 52162647 12% lli*
1892254 16435732 12% llvm-as*
6630877 54185542 12% llvm-db*
1811843 15667554 12% llvm-dis*
5752934 73995210 8% opt
556978 5153127 11% tblgen
I tried to explain this with two things: -On options. The existing
makefiles build without using -O. I've configure automake to use -O1 on
its compilations. That could still be the difference but I don't see how
it explains an order of magnitude. Another possibility was that I just
wasn't linking in all the static (pre-linked) object libraries in some
of the executables. But that doesn't explain llvm-as or llvm-db and I
have very meticulously made sure that what gets linked is the same as
the existing system (if I deviated, usually it didn't link). All the
programs run and produces the same --help out. That's significant
because if certain libraries were not being linked in, their command
line options wouldn't appear in the --help output.
The only thing I can think of is that the existing makefiles are doing
something weird that causes the bloat (probably in the debug info).
automake PROS/CONS
=================+ builds faster
+ builds smaller executables
+ we don't have to maintain it
+ new features/platforms are an upgrade away
+ standard makefile system known by lots of developers
+ GNU Makefile Standards compliance
+ rules are tried and tested and work well on lots of platforms
+ supports automatically creating a distribution (dist target)
+ supports automatically testing a distribution (distcheck target)
+ fast/standard install/uninstall targets
+ handles install/uninstall of scripts, headers, data, etc.
+ handles installcheck (checking programs after installation)
+ supports automated testing via dejagnu
+ completely groks libtool for building shared libs on lots of platforms
+ autoconf flags can determine nature of the build
+ supports C, C++, Fortran, Java, Obj-C, Lex, Yacc, Python
+ knows how to build texinfo & man page documentation
+ perfect dependency tracking (each compile updates the info)
+ automatically handles .exe (or other) extensions for executables
wherever its needed
+ automatically handles C/C++ tags/etags
+ supports building different targets with different compile options
+ can still be extended in any way GNU make can be
- have to specify each source input file (currently searching
for a workaround)
- completely dependent on gmake
- requires Perl (automake written in Perl)
existing PROS/CONS
=================+ its done and working
+ we have full control over anything
+ somewhat similar to the BSD makefile system
- completely dependent on gmake
- we must maintain it ourselves
- doesn't handle automatic distribution generation
- installs slowly
- testing support is adhoc/inconsistent
CURRENT THINGS TO FIX/FINISH IN AUTOMAKE RULES
=============================================
1. The default/only mode of output is VERBOSE.
2. Don't use more Makefile.am files than is necessary (e.g. SparcV9
should just have one, not 5)
3. Find a way to make it handle bytecode output better
4. Find a way to not maintain list of sources if the library
developer so chooses (i.e. assume all sources found)
5. Add support in test, examples, projects directories
Reid
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20041016/1e09efac/attachment.sig>
After puzzling about the size of the executables and the build times, I discovered (thanks Chris!) that I had compiled everything without debug symbols in the automake version. So, here's some revision from the first version of this email. The build times didn't change much (I guess I/O is cheap on my machine). The new "Build With Automake" times are 20m28.672s (elapsed), 18m1.900s (user) and 1m38.540s (system). The real change is in the size of the executables. Thew new values, while still smaller are much more reasonable. Previously the automake build and existing build were using different flags to the compiler. The results below are with the same flags (I double checked). Automake Existing Pct Name 16903982 46046545 37% analyze (bug in existing makefiles) 73084123 77679274 94% bugpoint 17638401 19137945 92% extract 37945217 47578060 80% gccas 31870129 34163210 93% gccld 56967280 60263187 95% llc 48570878 52162647 93% lli 15040029 16435732 92% llvm-as 50580919 54185542 93% llvm-db 14306895 15667554 91% llvm-dis 69413397 73995210 94% opt Sorry for the confusion. Reid. On Sat, 2004-10-16 at 23:25, Reid Spencer wrote:> Folks, > > I have completed the addition of automake makefiles to LLVM. All > libraries, tools, and runtime libs build now with automake. Note that > there are still many missing things in the automake support. Right now > it just builds the basic software. > > However, before I invest more time in it, I thought some comparison > would help us make some decisions about whether or not to proceed with > automake for the LLVM standard. There are costs and benefits on both > sides. > > BUILD/CONFIGURE TIMES > ================================================================> Item Elapsed User System > ----------------------------- ---------- ---------- ---------- > Configure With Automake 0m48.835s 0m22.700s 0m21.890s > Configure With Current System 0m37.939s 0m18.230s 0m16.980s > Build With Automake 20m29.057s 18m30.230s 1m28.080s > Build With Current System 30m56.017s 25m56.750s 3m15.790s > > This was recorded on a 2x2.4GHz Xeon RH9 Linux machine (same one used > for the optimized nightly test). The build was run with "gmake -j 1" in > both cases. The time includes the total time taken to build utils, lib, > tools, and runtime. > > Note that while the automake configure takes longer, automake builds the > software in about 1/3 less time than our existing system. A "gmake -j 3" > builds from scratch on my machine in under 10 minutes. Since building is > done much more frequently than configuring, this is a development win. > > EXECUTABLE SIZES > ===============> > Below is a list of the sizes of some of the larger/important executables > that LLVM builds. The bytesize of the Automake built version and the > Existing version are shown as well as a percentage. In every case the > automake executable is smaller, by an order of magnitude. > > AutoMake Existing Pct Program > -------- -------- ---- ------- > 2084426 46046545 5% analyze > 6118914 77679274 8% bugpoint* > 2038252 19137945 11% extract* > 3817030 47578060 8% gccas* > 3244568 34163210 9% gccld* > 5713818 60263187 9% llc* > 6446641 52162647 12% lli* > 1892254 16435732 12% llvm-as* > 6630877 54185542 12% llvm-db* > 1811843 15667554 12% llvm-dis* > 5752934 73995210 8% opt > 556978 5153127 11% tblgen > > I tried to explain this with two things: -On options. The existing > makefiles build without using -O. I've configure automake to use -O1 on > its compilations. That could still be the difference but I don't see how > it explains an order of magnitude. Another possibility was that I just > wasn't linking in all the static (pre-linked) object libraries in some > of the executables. But that doesn't explain llvm-as or llvm-db and I > have very meticulously made sure that what gets linked is the same as > the existing system (if I deviated, usually it didn't link). All the > programs run and produces the same --help out. That's significant > because if certain libraries were not being linked in, their command > line options wouldn't appear in the --help output. > > The only thing I can think of is that the existing makefiles are doing > something weird that causes the bloat (probably in the debug info). > > automake PROS/CONS > =================> + builds faster > + builds smaller executables > + we don't have to maintain it > + new features/platforms are an upgrade away > + standard makefile system known by lots of developers > + GNU Makefile Standards compliance > + rules are tried and tested and work well on lots of platforms > + supports automatically creating a distribution (dist target) > + supports automatically testing a distribution (distcheck target) > + fast/standard install/uninstall targets > + handles install/uninstall of scripts, headers, data, etc. > + handles installcheck (checking programs after installation) > + supports automated testing via dejagnu > + completely groks libtool for building shared libs on lots of platforms > + autoconf flags can determine nature of the build > + supports C, C++, Fortran, Java, Obj-C, Lex, Yacc, Python > + knows how to build texinfo & man page documentation > + perfect dependency tracking (each compile updates the info) > + automatically handles .exe (or other) extensions for executables > wherever its needed > + automatically handles C/C++ tags/etags > + supports building different targets with different compile options > + can still be extended in any way GNU make can be > - have to specify each source input file (currently searching > for a workaround) > - completely dependent on gmake > - requires Perl (automake written in Perl) > > existing PROS/CONS > =================> + its done and working > + we have full control over anything > + somewhat similar to the BSD makefile system > - completely dependent on gmake > - we must maintain it ourselves > - doesn't handle automatic distribution generation > - installs slowly > - testing support is adhoc/inconsistent > > CURRENT THINGS TO FIX/FINISH IN AUTOMAKE RULES > =============================================> > 1. The default/only mode of output is VERBOSE. > 2. Don't use more Makefile.am files than is necessary (e.g. SparcV9 > should just have one, not 5) > 3. Find a way to make it handle bytecode output better > 4. Find a way to not maintain list of sources if the library > developer so chooses (i.e. assume all sources found) > 5. Add support in test, examples, projects directories > > Reid >-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20041017/629577bb/attachment.sig>
One more update. The Makefile.am for analyze was wrong. It wasn't linking in the some of the passes. The new size is 56951088 which is in line with the other executables. Also, I have now completed a run of projects/llvm-test/MultiSource with the tools generated by automake. The only errors were for: TEST (llc) 'sgefa' FAILED! TEST (jit) 'sgefa' FAILED! TEST (jit) 'make_dparser' FAILED! TEST (llc) 'kc' FAILED! TEST (jit) 'anagram' FAILED! TEST (llc) 'mason' FAILED! TEST (cbe) 'mason' FAILED! TEST (jit) 'mason' FAILED! TEST (jit) 'pcompress2' FAILED! TEST (jit) 'make' FAILED! TEST (cbe) 'timberwolfmc' FAILED! TEST (jit) 'agrep' FAILED! Which isn't far off what the nightly test produces these days. Reid. On Sun, 2004-10-17 at 20:05, Reid Spencer wrote:> After puzzling about the size of the executables and the build times, > I discovered (thanks Chris!) that I had compiled everything without > debug symbols in the automake version. So, here's some revision from the > first version of this email. > > The build times didn't change much (I guess I/O is cheap on my machine). > The new "Build With Automake" times are 20m28.672s (elapsed), 18m1.900s > (user) and 1m38.540s (system). > > The real change is in the size of the executables. Thew new values, > while still smaller are much more reasonable. Previously the automake > build and existing build were using different flags to the compiler. The > results below are with the same flags (I double checked). > > Automake Existing Pct Name > 16903982 46046545 37% analyze (bug in existing makefiles) > 73084123 77679274 94% bugpoint > 17638401 19137945 92% extract > 37945217 47578060 80% gccas > 31870129 34163210 93% gccld > 56967280 60263187 95% llc > 48570878 52162647 93% lli > 15040029 16435732 92% llvm-as > 50580919 54185542 93% llvm-db > 14306895 15667554 91% llvm-dis > 69413397 73995210 94% opt > > Sorry for the confusion. > > Reid. > > On Sat, 2004-10-16 at 23:25, Reid Spencer wrote: > > Folks, > > > > I have completed the addition of automake makefiles to LLVM. All > > libraries, tools, and runtime libs build now with automake. Note that > > there are still many missing things in the automake support. Right now > > it just builds the basic software. > > > > However, before I invest more time in it, I thought some comparison > > would help us make some decisions about whether or not to proceed with > > automake for the LLVM standard. There are costs and benefits on both > > sides. > > > > BUILD/CONFIGURE TIMES > > ================================================================> > Item Elapsed User System > > ----------------------------- ---------- ---------- ---------- > > Configure With Automake 0m48.835s 0m22.700s 0m21.890s > > Configure With Current System 0m37.939s 0m18.230s 0m16.980s > > Build With Automake 20m29.057s 18m30.230s 1m28.080s > > Build With Current System 30m56.017s 25m56.750s 3m15.790s > > > > This was recorded on a 2x2.4GHz Xeon RH9 Linux machine (same one used > > for the optimized nightly test). The build was run with "gmake -j 1" in > > both cases. The time includes the total time taken to build utils, lib, > > tools, and runtime. > > > > Note that while the automake configure takes longer, automake builds the > > software in about 1/3 less time than our existing system. A "gmake -j 3" > > builds from scratch on my machine in under 10 minutes. Since building is > > done much more frequently than configuring, this is a development win. > > > > EXECUTABLE SIZES > > ===============> > > > Below is a list of the sizes of some of the larger/important executables > > that LLVM builds. The bytesize of the Automake built version and the > > Existing version are shown as well as a percentage. In every case the > > automake executable is smaller, by an order of magnitude. > > > > AutoMake Existing Pct Program > > -------- -------- ---- ------- > > 2084426 46046545 5% analyze > > 6118914 77679274 8% bugpoint* > > 2038252 19137945 11% extract* > > 3817030 47578060 8% gccas* > > 3244568 34163210 9% gccld* > > 5713818 60263187 9% llc* > > 6446641 52162647 12% lli* > > 1892254 16435732 12% llvm-as* > > 6630877 54185542 12% llvm-db* > > 1811843 15667554 12% llvm-dis* > > 5752934 73995210 8% opt > > 556978 5153127 11% tblgen > > > > I tried to explain this with two things: -On options. The existing > > makefiles build without using -O. I've configure automake to use -O1 on > > its compilations. That could still be the difference but I don't see how > > it explains an order of magnitude. Another possibility was that I just > > wasn't linking in all the static (pre-linked) object libraries in some > > of the executables. But that doesn't explain llvm-as or llvm-db and I > > have very meticulously made sure that what gets linked is the same as > > the existing system (if I deviated, usually it didn't link). All the > > programs run and produces the same --help out. That's significant > > because if certain libraries were not being linked in, their command > > line options wouldn't appear in the --help output. > > > > The only thing I can think of is that the existing makefiles are doing > > something weird that causes the bloat (probably in the debug info). > > > > automake PROS/CONS > > =================> > + builds faster > > + builds smaller executables > > + we don't have to maintain it > > + new features/platforms are an upgrade away > > + standard makefile system known by lots of developers > > + GNU Makefile Standards compliance > > + rules are tried and tested and work well on lots of platforms > > + supports automatically creating a distribution (dist target) > > + supports automatically testing a distribution (distcheck target) > > + fast/standard install/uninstall targets > > + handles install/uninstall of scripts, headers, data, etc. > > + handles installcheck (checking programs after installation) > > + supports automated testing via dejagnu > > + completely groks libtool for building shared libs on lots of platforms > > + autoconf flags can determine nature of the build > > + supports C, C++, Fortran, Java, Obj-C, Lex, Yacc, Python > > + knows how to build texinfo & man page documentation > > + perfect dependency tracking (each compile updates the info) > > + automatically handles .exe (or other) extensions for executables > > wherever its needed > > + automatically handles C/C++ tags/etags > > + supports building different targets with different compile options > > + can still be extended in any way GNU make can be > > - have to specify each source input file (currently searching > > for a workaround) > > - completely dependent on gmake > > - requires Perl (automake written in Perl) > > > > existing PROS/CONS > > =================> > + its done and working > > + we have full control over anything > > + somewhat similar to the BSD makefile system > > - completely dependent on gmake > > - we must maintain it ourselves > > - doesn't handle automatic distribution generation > > - installs slowly > > - testing support is adhoc/inconsistent > > > > CURRENT THINGS TO FIX/FINISH IN AUTOMAKE RULES > > =============================================> > > > 1. The default/only mode of output is VERBOSE. > > 2. Don't use more Makefile.am files than is necessary (e.g. SparcV9 > > should just have one, not 5) > > 3. Find a way to make it handle bytecode output better > > 4. Find a way to not maintain list of sources if the library > > developer so chooses (i.e. assume all sources found) > > 5. Add support in test, examples, projects directories > > > > Reid > > > > ______________________________________________________________________ > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20041017/89257b78/attachment.sig>