llvm dev - Nov 2018 - [cfe-dev] GN build roundtable summary; adding GN build files to the repo

There be dragons down the path of in-tree N > 1 build systems.

Here are just a few:
* Compiler bugs that turn out to be differences in how the compiler was
built due to differences in the two systems.
* Build breaks caused by lag-time between synchronization of the N build
systems.
* Development overhead of new features requiring devs to test it in both
build configurations or risk build breaks.
* Maintainer issues for both build systems when devs leave and users report
bugs.

This will inevitably lead to another community effort to deprecate one of
the two build systems, similar to what happened in 2013 with autoconf.
http://llvm.1065342.n5.nabble.com/Deprecating-autoconf-make-td57924.html

GN is the current build darling of Google. However, on my Red Hat and
Ubuntu hosts there are no packages to install and others have pointed out
certain architectures aren't supported with GN today. Until gn is a 'dnf
install gn' or 'apt install gn' away this makes the build
dependencies on
LLVM more onerous.

I'm also a bit hesitant because the culture at Google seems to quickly
deprecate services and tools for "the-next-new-shiny".  I remember
when
Chromium and the AOSP used makefiles and moved to GYP. And now GN is
replacing GYP in several projects. That's two build system overhauls in the
span of about 4 years.

I would rather have the discussion of the pros/cons of fully replacing
CMake and all that would entail for a complex project like LLVM.

On Thu, Nov 1, 2018 at 11:21 AM John Paul Adrian Glaubitz via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> On 11/1/18 1:29 AM, Petr Hosek wrote:
> > I'm one of GN maintainers, I'd be happy to review to review
and
> > patches for other systems and architectures. We accepted AIX port a
> > few months ago:
> >
> https://gn.googlesource.com/gn/+/499868c6781dfb87dcefafd8f68dc578f3b70b44
>
> Fair enough. I just had the experience with SKIA where my patches were
> outright rejected with the answer "We don't care about
big-endian".
>
> This was also the first time I had contact with GN, I wanted to build
> SKIA from source on a non-x86 target and eventually gave up after half
> an hour or so.
>
> >> Oh, and btw, any serious Linux distribution would outright reject
to
> package
> >> something that involves downloading another chroot to be able to
build
> it,
> >> build daemons are - by definition - offline.
> >
> > We're downloading the sysroot we use to build against to avoid
> > dependencies on anything on the bot itself. I agree with you
> > completely that this is something that build shouldn't do, and I
plan
> > on moving that step to the bot script.
>
> Well, there is no need to tie this into the build system directly,
that's
> what Travis-CI or Jenkins are for. I'm just getting a bad feeling in my
> stomach if the solution for building a tool from source is to download
> a complete build environment instead of using tools on the target system
> to achieve that.
>
> > Until a few months ago, GN was part of Chromium and was relying on its
> > infrastructure, build and source. We've decided to split GN into a
> > separate project because it's being adopted by other projects and
> > users outside of Chromium. We're still dealing with fallout of
that
> > separation and resulting cleanup. I hope that eventually building GN
> > will be as simple and easy as Ninja itself.
>
> Ok, then I hope that LLVM will be buildable with cmake until GN is
> mature enough that I can build it from source on any architecture
> on Linux without me jumping through loops.
>
> Thanks,
> Adrian
>
> --
>  .''`.  John Paul Adrian Glaubitz
> : :' :  Debian Developer - glaubitz at debian.org
> `. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
>   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181101/5b35a83f/attachment.html>

On 01.11.18 17:44, Brian M. Rzycki via llvm-dev wrote:
> There be dragons down the path of in-tree N > 1 build systems.
> 
> Here are just a few:
> * Compiler bugs that turn out to be differences in how the compiler was 
> built due to differences in the two systems.
> * Build breaks caused by lag-time between synchronization of the N build 
> systems.
> * Development overhead of new features requiring devs to test it in both 
> build configurations or risk build breaks.
> * Maintainer issues for both build systems when devs leave and users 
> report bugs.
> 
> This will inevitably lead to another community effort to deprecate one 
> of the two build systems, similar to what happened in 2013 with autoconf.
> http://llvm.1065342.n5.nabble.com/Deprecating-autoconf-make-td57924.html
> 
> GN is the current build darling of Google. However, on my Red Hat and 
> Ubuntu hosts there are no packages to install and others have pointed 
> out certain architectures aren't supported with GN today. Until gn is a
> 'dnf install gn' or 'apt install gn' away this makes the
build
> dependencies on LLVM more onerous.
> 
> I'm also a bit hesitant because the culture at Google seems to quickly 
> deprecate services and tools for "the-next-new-shiny".  I
remember when
> Chromium and the AOSP used makefiles and moved to GYP. And now GN is 
> replacing GYP in several projects. That's two build system overhauls in
> the span of about 4 years.
+1 for not jumping on Google's NIH train.

Don't they also have Bazel, for example? I'm extremely skeptical about 
jumping on their latest shiny thing which they're likely to drop again 
soon enough, given their track record.

Cheers,
Nicolai



> 
> I would rather have the discussion of the pros/cons of fully replacing 
> CMake and all that would entail for a complex project like LLVM.
> 
> On Thu, Nov 1, 2018 at 11:21 AM John Paul Adrian Glaubitz via cfe-dev 
> <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
wrote:
> 
>     On 11/1/18 1:29 AM, Petr Hosek wrote:
>      > I'm one of GN maintainers, I'd be happy to review to
review and
>      > patches for other systems and architectures. We accepted AIX port
a
>      > few months ago:
>      >
>    
https://gn.googlesource.com/gn/+/499868c6781dfb87dcefafd8f68dc578f3b70b44
> 
>     Fair enough. I just had the experience with SKIA where my patches were
>     outright rejected with the answer "We don't care about
big-endian".
> 
>     This was also the first time I had contact with GN, I wanted to build
>     SKIA from source on a non-x86 target and eventually gave up after half
>     an hour or so.
> 
>      >> Oh, and btw, any serious Linux distribution would outright
>     reject to package
>      >> something that involves downloading another chroot to be able
to
>     build it,
>      >> build daemons are - by definition - offline.
>      >
>      > We're downloading the sysroot we use to build against to
avoid
>      > dependencies on anything on the bot itself. I agree with you
>      > completely that this is something that build shouldn't do,
and I plan
>      > on moving that step to the bot script.
> 
>     Well, there is no need to tie this into the build system directly,
>     that's
>     what Travis-CI or Jenkins are for. I'm just getting a bad feeling
in my
>     stomach if the solution for building a tool from source is to download
>     a complete build environment instead of using tools on the target
system
>     to achieve that.
> 
>      > Until a few months ago, GN was part of Chromium and was relying
>     on its
>      > infrastructure, build and source. We've decided to split GN
into a
>      > separate project because it's being adopted by other projects
and
>      > users outside of Chromium. We're still dealing with fallout
of that
>      > separation and resulting cleanup. I hope that eventually building
GN
>      > will be as simple and easy as Ninja itself.
> 
>     Ok, then I hope that LLVM will be buildable with cmake until GN is
>     mature enough that I can build it from source on any architecture
>     on Linux without me jumping through loops.
> 
>     Thanks,
>     Adrian
> 
>     -- 
>       .''`.  John Paul Adrian Glaubitz
>     : :' :  Debian Developer - glaubitz at debian.org
>     <mailto:glaubitz at debian.org>
>     `. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
>     <mailto:glaubitz at physik.fu-berlin.de>
>        `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
> 
>     _______________________________________________
>     cfe-dev mailing list
>     cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
-- 
Lerne, wie die Welt wirklich ist,
Aber vergiss niemals, wie sie sein sollte.

Hi!

On 11/1/18 5:44 PM, Brian M. Rzycki via llvm-dev wrote:
> GN is the current build darling of Google. However, on my Red Hat and
Ubuntu hosts there are no packages to install and others have pointed out
certain architectures aren't supported with GN today. Until gn is a 'dnf
install gn' or 'apt install gn' away this makes the build
dependencies on LLVM more onerous.
Jepp, that's basically my main argument. Apparently, the current plan seems
to be
"Let's integrate it first, then we'll resolve the issues that GN
currently has
 some time later."
> I'm also a bit hesitant because the culture at Google seems to quickly
deprecate services and tools for "the-next-new-shiny".  I remember
when Chromium and the AOSP used makefiles and moved to GYP. And now GN is
replacing GYP in several projects. That's two build system overhauls in the
span of about 4 years.
That's really short. I would fear that Google drops GN only short time later
and
then we are left with something in LLVM that is unmaintained and unsupported by
upstream.
> I would rather have the discussion of the pros/cons of fully replacing
CMake and all that would entail for a complex project like LLVM.
And keeping in mind that LLVM is an extremely important project that a lot of
downstream
projects are building on because of the fact that a lot of languages like Rust
and Julia
are relying on LLVM as their code-generating backend. So, if you break the build
system,
you will get a lot of angry downstream users.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913