[LLVMdev] Re: [llvm-commits] CVS: */Makefile.am

John Criswell
Mon, 11 Oct 2004 14:39:18 -0000

Reid Spencer wrote:
> On Sun, 2004-10-10, Misha Brukman asked "Why can't we use
wildcards
> instead of listing all the sources" and then wrote in response to my
> reply:
>
>>On Sun, Oct 10, 2004 at 04:40:48PM -0700, Reid Spencer wrote:
>>
>>>http://www.gnu.org/software/automake/manual/html_mono/automake.html#wildcards
>>>
>>>is your answer.
>>
>>I think their "answer" is lame; we have already sacrificed
portability
>>because we are using enough of gmake-only functionality because we
>>cannot depend on the system "make" being anything useful, yet
gmake is
>>portable, and hence, the "standard".
>
>
> I agree completely.
>
>
>>The thing about "distribute only what you list", well...
maybe, but the
>>way "WE" do distribution is (I believe) checkout, build, test,
make
>>clean, make dist.  That way, you *know* that what you're
distributing
>>actually works.  But I'm not the distro person, that's John. :)

Typically, I do the following:

Checkout LLVM with release tag.
Make Distro.
Unpackage distro.
Build Distro.
Test Distro.

Yes, I'm paranoid.
:)

>
>
> Well, automake changes all that. Right now it takes John 1-2 weeks to do
> a "distro". automake can produce a VALIDATED distribution
automatically
> with only CPU time involved. The "distcheck" target builds a
> distribution candidate, produces a tar.gz of it, unpacks it, rebuilds
> it, and if everything is okay, the tar.gz is good to go. This will find
> ALL KINDS of subtle errors and a big part of getting the distribution
> right is setting up the _SOURCES variables to list things correctly.

FWIW, I don't believe that the build system for LLVM proper is the
problem.  I think the problems stem from the following:

o) The llvm-gcc binary distributions take awhile due to the following
complications:
	o After building it, I put the build into a cfrontend directory and add
copyright files and fixheader scripts.
	o For systems in which I don't want to deal with header file licensing
issues (i.e. Solaris and MacOS X), I remove all of the system headers
that GCC fixed when I built it.
	o Building llvm-gcc and LLVM is sort of a chicken and egg problem.  I
have to build LLVM, then llvm-gcc, and then build llvm/runtime and
install it into the llvm-gcc binary tree.  And if something gets fixed
in LLVM, it might change the libraries created by llvm/runtime, so
keeping everything in sync is kind of a pain.

o) Problems which don't affect day to day work typically don't get fixed
until release time.  Despite many people's efforts to fix bugs
immediently when they show up in the nightly tester, problems slip
through because our day to day priorities don't allow us to fix every
problem.

Part of that two weeks is one week worth of hurriedly fixing bugs that
we've neglected (because for releases, we want everything that worked in
the previous release to work in this current release).

Chris has recommended that we schedule some "cleaning up" time more
regularly to relieve this burden.

o) We support 3 platforms in-house (one of which gets neglected), and
each new platform that I can build increases the problems caused by the
two things above.

I believe the two best things that we can do to shorten releases is the
following:

1. Re-arrange the LLVM and LLVM-GCC builds to build in one fell swoop.
I checkout out, I type make, and it all builds.  This requires moving
parts of llvm/runtime into llvm-gcc, or making llvm-gcc an LLVM project,
or some other approach.

2. Schedule some time at least a month before the release to start
polishing LLVM and fix regressions that haven't already been tackled,
per Chris's suggestion.

I don't believe automake will make releases go any faster because it
doesn't solve the 2 problems listed above.  The LLVM core works pretty
well; it's the interaction with llvm-gcc that is problematic.

That said, I'm not against automake.  I just want to point out what I
think the release bottlenecks are.

-- John T.

--
*********************************************************************
* John T. Criswell                         Email: criswell at uiuc.edu *
* Research Programmer                                               *
* University of Illinois at Urbana-Champaign                        *
*                                                                   *
* "It's today!" said Piglet. "My favorite day," said
Pooh.          *
*********************************************************************