llvm dev - Nov 2012 - [LLVMdev] Last Known Good Builds?

Hi,

I have several times over the past three months tried to retrieve the trunk
version of LLVM/Clang/Compiler-RT from Subversion and then tried to build
it on Windows: Without success.  Also, I occasionally get emails from
people who ask me why LLVM/Windows does not build (they find my name in the
mailing list archive).

This has made me ponder the issue and I came up with the following scheme
to ensure that there is a high probability that we Windows users can
actually build the most recent version of LLVM:

1. Every night, an automated build builds LLVM & Co. on all the supported
host platforms and runs the test suite to completion.
2. IF the automated build succceds on ALL supported host platforms AND the
test suite succeeds, the build is marked as "Last Known Good" and
automatically published to the website so that people can download it
without using Subversion.  There should always only be one and only one
Last Known Good (LKG) build - which is sort of equivalent to a "daily
tarball", albeit it is known to build properly on all platforms.

I believe step 1 is already in place, but perhaps the project's build
master, or I, can put together some Python scripts to automate the process
of making a LKG build.

Why is the LKG build so important?  Because in software development in
general, you want to reduce the "granularity" of the process as much
as
possible, approximating a kurve rather than a series of steps.  If people
adopt LLVM v3.1 and then have to wait one year for another update (v3.2) in
which LOTS and LOTS of things have changed, they are in for a tough ride
when they want to upgrade to the new version.  If, on the other hand, they
can get a bi-weekly LKG build (assuming the entire build process on all
platforms only succeeds twice a week), they can adopt their code in very
small steps.


Cheers and have a great day!

Mikael Lyngvig
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121104/910c8adb/attachment.html>

> Why is the LKG build so important?  Because in software development in
> general, you want to reduce the "granularity" of the process as
much as
> possible, approximating a kurve rather than a series of steps.  If people
> adopt LLVM v3.1 and then have to wait one year for another update (v3.2) in
> which LOTS and LOTS of things have changed, they are in for a tough ride
> when they want to upgrade to the new version.  If, on the other hand, they
> can get a bi-weekly LKG build (assuming the entire build process on all
> platforms only succeeds twice a week), they can adopt their code in very
> small steps.
Do you have any information about how people are setting up their
projects? I ask because there are different ways to set up a project
to depend on LLVM.

Probably the most robust is how the Rust language
<https://github.com/mozilla/rust> does it, which is to set up a git
submodule which is pinned on a particular git commit; the revision
that it is pinned on is actually for a fork
<https://github.com/brson/llvm> of LLVM's main development which the
Rust developers have made specifically for this purpose. I don't
really track Rust, but from a quick browse, it looks like what they
are doing is to periodically sync with upstream or cherry-pick
specific bugfixes; they also put in some little fixes of their own. My
impression is that this is really robust and flexible, since it allows
them to sync with upstream at any granularity they want.

Another possibility that a lot of projects do is to just rely on the
"installed" version of LLVM, and just say 3.0+ or 3.1+. As you
mentioned, this really is not very good, since LLVM really has no
backwards compatibility policy they are in for a really rough ride
when the next version comes out.

One benefit of the Rust-style setup is that it allows you to easily
set up a job on your own servers that automatically pulls the last
green LLVM release (the buildbots have JSON API's which should allow
extracting this) and merges it in and then tries to build your project
with that. That way, you can stay up to date in a relatively automated
fashion and can ping the list for help with migrating to new APIs near
the time that the APIs are changed and not months later. Personally I
think this is a better alternative than an LKG tarball.

Overall, what you are hitting against is just a product of how LLVM is
developed (very fast moving, straight-line SVN, highly-volatile trunk,
with no backwards compatibility). Even though the code is nicely
modularized and loosely-coupled so that in theory it is very nice and
modular and easy to use as a library, the reality is that there is a
very strong *developer coupling*, in that you have to really be aware
of what is happening in the LLVM codebase or else your project will
bitrot extremely quickly. Since LLVM doesn't maintain any kind of
"migration guide" for its changes, you basically have to be an LLVM
developer to fight the bitrot, or else have some kind of automated
thing which will allow you to squawk on the mailing list for migration
help while the change is still fresh.

-- Sean Silva

On Sun, Nov 4, 2012 at 4:33 AM, Mikael Lyngvig <mikael at lyngvig.org>
wrote:
> Hi,
>
> I have several times over the past three months tried to retrieve the trunk
> version of LLVM/Clang/Compiler-RT from Subversion and then tried to build
it
> on Windows: Without success.  Also, I occasionally get emails from people
> who ask me why LLVM/Windows does not build (they find my name in the
mailing
> list archive).
>
> This has made me ponder the issue and I came up with the following scheme
to
> ensure that there is a high probability that we Windows users can actually
> build the most recent version of LLVM:
>
> 1. Every night, an automated build builds LLVM & Co. on all the
supported
> host platforms and runs the test suite to completion.
> 2. IF the automated build succceds on ALL supported host platforms AND the
> test suite succeeds, the build is marked as "Last Known Good" and
> automatically published to the website so that people can download it
> without using Subversion.  There should always only be one and only one
Last
> Known Good (LKG) build - which is sort of equivalent to a "daily
tarball",
> albeit it is known to build properly on all platforms.
>
> I believe step 1 is already in place, but perhaps the project's build
> master, or I, can put together some Python scripts to automate the process
> of making a LKG build.
>
> Why is the LKG build so important?  Because in software development in
> general, you want to reduce the "granularity" of the process as
much as
> possible, approximating a kurve rather than a series of steps.  If people
> adopt LLVM v3.1 and then have to wait one year for another update (v3.2) in
> which LOTS and LOTS of things have changed, they are in for a tough ride
> when they want to upgrade to the new version.  If, on the other hand, they
> can get a bi-weekly LKG build (assuming the entire build process on all
> platforms only succeeds twice a week), they can adopt their code in very
> small steps.
>
>
> Cheers and have a great day!
>
> Mikael Lyngvig
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

| Do you have any information about how people are setting up their
| projects? I ask because there are different ways to set up a project
| to depend on LLVM.

No, I don't have that kind of information.  All I know is that it is quite
difficult to find a working version on the trunk.  I think this especially
hit those that are new to LLVM and yet want to play around with the
bleeding edge version.

I think you are right about extracting the last green build using the JSON
API instead of a LKG tarball.  I didn't know it existed and hardly know
what it means.  JSON is some Java technology?  Is there any documentation
of this anywhere?

Perhaps somebody will volunteer to write up a few pages on how to extract
the last green build like you say?  That would be most helpful.

2012/11/5 Sean Silva <silvas at purdue.edu>
> > Why is the LKG build so important?  Because in software development in
> > general, you want to reduce the "granularity" of the process
as much as
> > possible, approximating a kurve rather than a series of steps.  If
people
> > adopt LLVM v3.1 and then have to wait one year for another update
(v3.2)
> in
> > which LOTS and LOTS of things have changed, they are in for a tough
ride
> > when they want to upgrade to the new version.  If, on the other hand,
> they
> > can get a bi-weekly LKG build (assuming the entire build process on
all
> > platforms only succeeds twice a week), they can adopt their code in
very
> > small steps.
>
> Do you have any information about how people are setting up their
> projects? I ask because there are different ways to set up a project
> to depend on LLVM.
>
> Probably the most robust is how the Rust language
> <https://github.com/mozilla/rust> does it, which is to set up a git
> submodule which is pinned on a particular git commit; the revision
> that it is pinned on is actually for a fork
> <https://github.com/brson/llvm> of LLVM's main development which
the
> Rust developers have made specifically for this purpose. I don't
> really track Rust, but from a quick browse, it looks like what they
> are doing is to periodically sync with upstream or cherry-pick
> specific bugfixes; they also put in some little fixes of their own. My
> impression is that this is really robust and flexible, since it allows
> them to sync with upstream at any granularity they want.
>
> Another possibility that a lot of projects do is to just rely on the
> "installed" version of LLVM, and just say 3.0+ or 3.1+. As you
> mentioned, this really is not very good, since LLVM really has no
> backwards compatibility policy they are in for a really rough ride
> when the next version comes out.
>
> One benefit of the Rust-style setup is that it allows you to easily
> set up a job on your own servers that automatically pulls the last
> green LLVM release (the buildbots have JSON API's which should allow
> extracting this) and merges it in and then tries to build your project
> with that. That way, you can stay up to date in a relatively automated
> fashion and can ping the list for help with migrating to new APIs near
> the time that the APIs are changed and not months later. Personally I
> think this is a better alternative than an LKG tarball.
>
> Overall, what you are hitting against is just a product of how LLVM is
> developed (very fast moving, straight-line SVN, highly-volatile trunk,
> with no backwards compatibility). Even though the code is nicely
> modularized and loosely-coupled so that in theory it is very nice and
> modular and easy to use as a library, the reality is that there is a
> very strong *developer coupling*, in that you have to really be aware
> of what is happening in the LLVM codebase or else your project will
> bitrot extremely quickly. Since LLVM doesn't maintain any kind of
> "migration guide" for its changes, you basically have to be an
LLVM
> developer to fight the bitrot, or else have some kind of automated
> thing which will allow you to squawk on the mailing list for migration
> help while the change is still fresh.
>
> -- Sean Silva
>
> On Sun, Nov 4, 2012 at 4:33 AM, Mikael Lyngvig <mikael at
lyngvig.org> wrote:
> > Hi,
> >
> > I have several times over the past three months tried to retrieve the
> trunk
> > version of LLVM/Clang/Compiler-RT from Subversion and then tried to
> build it
> > on Windows: Without success.  Also, I occasionally get emails from
people
> > who ask me why LLVM/Windows does not build (they find my name in the
> mailing
> > list archive).
> >
> > This has made me ponder the issue and I came up with the following
> scheme to
> > ensure that there is a high probability that we Windows users can
> actually
> > build the most recent version of LLVM:
> >
> > 1. Every night, an automated build builds LLVM & Co. on all the
supported
> > host platforms and runs the test suite to completion.
> > 2. IF the automated build succceds on ALL supported host platforms AND
> the
> > test suite succeeds, the build is marked as "Last Known
Good" and
> > automatically published to the website so that people can download it
> > without using Subversion.  There should always only be one and only
one
> Last
> > Known Good (LKG) build - which is sort of equivalent to a "daily
> tarball",
> > albeit it is known to build properly on all platforms.
> >
> > I believe step 1 is already in place, but perhaps the project's
build
> > master, or I, can put together some Python scripts to automate the
> process
> > of making a LKG build.
> >
> > Why is the LKG build so important?  Because in software development in
> > general, you want to reduce the "granularity" of the process
as much as
> > possible, approximating a kurve rather than a series of steps.  If
people
> > adopt LLVM v3.1 and then have to wait one year for another update
(v3.2)
> in
> > which LOTS and LOTS of things have changed, they are in for a tough
ride
> > when they want to upgrade to the new version.  If, on the other hand,
> they
> > can get a bi-weekly LKG build (assuming the entire build process on
all
> > platforms only succeeds twice a week), they can adopt their code in
very
> > small steps.
> >
> >
> > Cheers and have a great day!
> >
> > Mikael Lyngvig
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121105/794647b7/attachment.html>