llvm dev - Nov 2018 - [cfe-dev] GN build roundtable summary; adding GN build files to the repo

Any easy way to do this as some kind of overlay, so they GN files wouldn't
live in the LLVM repository, but in a separate one?

(this might avoid some of the community discussion - though would perhaps
still likely have the issue I see as most significant: With a sufficient
number of developers using GN, the rate of build breaks due to those
developers missing CMake file updates might rise to be a bit of a drag on
the LLVM project - though I don't think that's likely to ever be a huge
deal, just an annoyance)

On Wed, Oct 31, 2018 at 11:19 AM Nico Weber via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> Hi,
>
> first things first: If you're happy with cmake, you can stop reading
now.
> Nobody is proposing that LLVM moves off cmake, and nobody is proposing
> anything that's causing people using cmake more work.
>
> At the LLVM conference, I gave a lightning talk [1] about using GN [2] to
> build LLVM and clang. cmake is great for many use cases, but in my opinion
> local day-to-day development isn't one of them. So I wrote GN build
files
> for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld`
> build everything needed for these three test suites and that all tests
> pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You
> can see them at [3].
>
> I had been worried that it would be a lot of work to keep the build files
> up to date, but I've been using this for all my LLVM/clang/lld
development
> the last 8 months, and it turned out to not be a big problem -- LLVM's
> build files don't change very often, and GN build files are a pleasure
to
> work with in my opinion.
>
> I gave the lightning talk just to talk about my personal workflow, but
> there was a lot of interest. We had a roundtable on the next day, and about
> 20 people said they'd be interested in using this for their development
> too. The main request was that the .gn files are checked in upstream, so
> that we can collaborate on keeping them working. Two to three orgs even
> said they'd be interested in moving their buildbots to GN.
>
> As mentioned at the top, the intention here is not to replace cmake, only
> to offer an opt-in alternative for people who are interested in it. Keeping
> the GN build going would be the responsibility of people using it, not of
> the general LLVM community. If this fails to find use and bitrots, we can
> easily remove it again.
>
> Are there any concerns with checking in GN files? I've put some initial
> docs for the GN build at https://reviews.llvm.org/D53944 ; it describes
> what the GN build is and is not, what its advantages are (speed and easier
> configurability), and some points about the philosophy behind the LLVM GN
> build.
>
> If folks are generally ok with GN files living in-tree, I'll start
sending
> patches for gradually adding gn files through the regular review process.
>
> If having a BUILD.gn file in every directory being confusing is a concern,
> GN has the concept of a "secondary tree" so that all GN files
could be
> below llvm/gn/tree/{llvm,clang,lld,...}.
>
> Cheers,
> Nico
>
> 1: https://llvm.org/devmtg/2018-10/talk-abstracts.html#lt2
> 2: https://gn.googlesource.com/gn , https://is.gd/gn_intro
> 3:
>
https://github.com/llvm-project/llvm-project-20170507/compare/master...nico:gn
> , click "Files Changed" to see the GN files.
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181101/ce9e194b/attachment.html>

I think this is how, after moving to a monorepo, it may be feasible to
get this in a separate fork.

Granted the Git Monorepo + GitHub happens, we can even make it so that
the GN build is a branch on the official git repository, which can
track the mainline development.
On Fri, Nov 2, 2018 at 3:49 AM David Blaikie via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Any easy way to do this as some kind of overlay, so they GN files
wouldn't live in the LLVM repository, but in a separate one?
>
> (this might avoid some of the community discussion - though would perhaps
still likely have the issue I see as most significant: With a sufficient number
of developers using GN, the rate of build breaks due to those developers missing
CMake file updates might rise to be a bit of a drag on the LLVM project - though
I don't think that's likely to ever be a huge deal, just an annoyance)
>
> On Wed, Oct 31, 2018 at 11:19 AM Nico Weber via cfe-dev <cfe-dev at
lists.llvm.org> wrote:
>>
>> Hi,
>>
>> first things first: If you're happy with cmake, you can stop
reading now. Nobody is proposing that LLVM moves off cmake, and nobody is
proposing anything that's causing people using cmake more work.
>>
>> At the LLVM conference, I gave a lightning talk [1] about using GN [2]
to build LLVM and clang. cmake is great for many use cases, but in my opinion
local day-to-day development isn't one of them. So I wrote GN build files
for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld`
build everything needed for these three test suites and that all tests pass.
This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see
them at [3].
>>
>> I had been worried that it would be a lot of work to keep the build
files up to date, but I've been using this for all my LLVM/clang/lld
development the last 8 months, and it turned out to not be a big problem --
LLVM's build files don't change very often, and GN build files are a
pleasure to work with in my opinion.
>>
>> I gave the lightning talk just to talk about my personal workflow, but
there was a lot of interest. We had a roundtable on the next day, and about 20
people said they'd be interested in using this for their development too.
The main request was that the .gn files are checked in upstream, so that we can
collaborate on keeping them working. Two to three orgs even said they'd be
interested in moving their buildbots to GN.
>>
>> As mentioned at the top, the intention here is not to replace cmake,
only to offer an opt-in alternative for people who are interested in it. Keeping
the GN build going would be the responsibility of people using it, not of the
general LLVM community. If this fails to find use and bitrots, we can easily
remove it again.
>>
>> Are there any concerns with checking in GN files? I've put some
initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes
what the GN build is and is not, what its advantages are (speed and easier
configurability), and some points about the philosophy behind the LLVM GN build.
>>
>> If folks are generally ok with GN files living in-tree, I'll start
sending patches for gradually adding gn files through the regular review
process.
>>
>> If having a BUILD.gn file in every directory being confusing is a
concern, GN has the concept of a "secondary tree" so that all GN files
could be below llvm/gn/tree/{llvm,clang,lld,...}.
>>
>> Cheers,
>> Nico
>>
>> 1: https://llvm.org/devmtg/2018-10/talk-abstracts.html#lt2
>> 2: https://gn.googlesource.com/gn , https://is.gd/gn_intro
>> 3:
https://github.com/llvm-project/llvm-project-20170507/compare/master...nico:gn ,
click "Files Changed" to see the GN files.
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


-- 
Dean

If I read this correctly, there isn't much opposition to landing the gn
files as long as it's very clear that regular devs aren't supposed to
update them and that it's clear that they're experimental

The main concerns I've heard so far:

- Having two build systems is confusing. I can see this, but I think
putting the gn files below llvm/experimental/gn (instead of right into the
source, like I currently have) and updating GN.rst to explicitly say
"Reviewers should not ask patch authors to update gn files in addition to
the cmake files" will address this.

- Having a few people care about the GN build means these people won't
improve the cmake build. I think this has some merit (but I did clean up
parts of the cmake build a bit while reading all our cmake code to create
the gn build for example, and I have several ideas about improving the
cmake build I want to implement at some point; and it's also not a given
that the folks who may or may not end up working on the GN build would have
worked on the cmake build). However, this is true independently of where
the GN build files are stored.

- GN isn't a cmake replacement, for distro reasons and whatnot. I
wholeheartedly agree with this. Maybe GN will become better here, but cmake
is ubiquitous _today_, and it's backed by a company who's interested in
keeping it around, so it will be around for a long time. I think this is
cmake's biggest strength. That's why I'm not proposing on replacing
the
cmake build. If the GN build at some point is way better (compiler-rt,
lldb, etc added; out-of-tree build support; GN itself gets a real distro
story; ...) then this _may_ be different in a few years, at which point we
could reconsider. I think this is unlikely to happen.

(I still don't see that any of these problems are being solved by having
this in an overlay or a separate fork.)

And again, It's a real possibility that we check this in and it turns out
the people who are interested in don't feel it's all that useful, it
bitrots, and we delete it again. That's cool, no harm done. I think we
should have a very high standard for replacing the supported build system
(cmake), but I think it's cool to have a low bar for people experimenting
with unsupported build systems, as long as they get deleted if not used. If
someone wanted to a, say, llvm/experimental/meson to see how that would
feel, I think that'd be super cool too for example.

On Fri, Nov 2, 2018 at 2:06 AM Dean Michael Berris <dean.berris at
gmail.com>
wrote:
> I think this is how, after moving to a monorepo, it may be feasible to
> get this in a separate fork.
>
> Granted the Git Monorepo + GitHub happens, we can even make it so that
> the GN build is a branch on the official git repository, which can
> track the mainline development.
> On Fri, Nov 2, 2018 at 3:49 AM David Blaikie via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > Any easy way to do this as some kind of overlay, so they GN files
> wouldn't live in the LLVM repository, but in a separate one?
> >
> > (this might avoid some of the community discussion - though would
> perhaps still likely have the issue I see as most significant: With a
> sufficient number of developers using GN, the rate of build breaks due to
> those developers missing CMake file updates might rise to be a bit of a
> drag on the LLVM project - though I don't think that's likely to
ever be a
> huge deal, just an annoyance)
> >
> > On Wed, Oct 31, 2018 at 11:19 AM Nico Weber via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
> >>
> >> Hi,
> >>
> >> first things first: If you're happy with cmake, you can stop
reading
> now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing
> anything that's causing people using cmake more work.
> >>
> >> At the LLVM conference, I gave a lightning talk [1] about using GN
[2]
> to build LLVM and clang. cmake is great for many use cases, but in my
> opinion local day-to-day development isn't one of them. So I wrote GN
build
> files for LLVM and clang, enough to make `ninja check-llvm check-clang
> check-lld` build everything needed for these three test suites and that all
> tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM,
> AArch64. You can see them at [3].
> >>
> >> I had been worried that it would be a lot of work to keep the
build
> files up to date, but I've been using this for all my LLVM/clang/lld
> development the last 8 months, and it turned out to not be a big problem --
> LLVM's build files don't change very often, and GN build files are
a
> pleasure to work with in my opinion.
> >>
> >> I gave the lightning talk just to talk about my personal workflow,
but
> there was a lot of interest. We had a roundtable on the next day, and about
> 20 people said they'd be interested in using this for their development
> too. The main request was that the .gn files are checked in upstream, so
> that we can collaborate on keeping them working. Two to three orgs even
> said they'd be interested in moving their buildbots to GN.
> >>
> >> As mentioned at the top, the intention here is not to replace
cmake,
> only to offer an opt-in alternative for people who are interested in it.
> Keeping the GN build going would be the responsibility of people using it,
> not of the general LLVM community. If this fails to find use and bitrots,
> we can easily remove it again.
> >>
> >> Are there any concerns with checking in GN files? I've put
some initial
> docs for the GN build at https://reviews.llvm.org/D53944 ; it describes
> what the GN build is and is not, what its advantages are (speed and easier
> configurability), and some points about the philosophy behind the LLVM GN
> build.
> >>
> >> If folks are generally ok with GN files living in-tree, I'll
start
> sending patches for gradually adding gn files through the regular review
> process.
> >>
> >> If having a BUILD.gn file in every directory being confusing is a
> concern, GN has the concept of a "secondary tree" so that all GN
files
> could be below llvm/gn/tree/{llvm,clang,lld,...}.
> >>
> >> Cheers,
> >> Nico
> >>
> >> 1: https://llvm.org/devmtg/2018-10/talk-abstracts.html#lt2
> >> 2: https://gn.googlesource.com/gn , https://is.gd/gn_intro
> >> 3:
>
https://github.com/llvm-project/llvm-project-20170507/compare/master...nico:gn
> , click "Files Changed" to see the GN files.
> >> _______________________________________________
> >> cfe-dev mailing list
> >> cfe-dev at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> --
> Dean
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181105/29736396/attachment.html>