llvm dev - Nov 2018 - [cfe-dev] GN build roundtable summary; adding GN build files to the repo

Awesome. I'm happy with moving the .rst file into that directory and not
have it on the public website. I'll try to make a patch that lands enough
scaffolding to build `not` in the next few days, and then I'll land the
other build files I have through the regular build process after that.
Unless someone feels strongly, I'll go with Justin's suggestion of
llvm/utils/gn/...

On Tue, Nov 6, 2018 at 12:20 PM David Blaikie <dblaikie at gmail.com>
wrote:
> Yeah, I don't feel super strongly about the name or location if
it's a
> single directory entry point. Not sure if/where it should be documented (if
> it needs web documentation - rather than or in addition to a README in that
> directory) - don't really want to muddy the waters in the official docs
as
> others have mentioned. I'm OK with it going in-tree under those
conditions,
> FWIW.
>
> On Tue, Nov 6, 2018 at 3:10 AM Nico Weber via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> The value in having them somewhere in-tree is that it's easier for
people
>> collaborate on these files, and it's way lower setup overhead if
someone
>> wants to try it out. If people prefer llvm/util over llvm/experimental,
>> that's fine with me.
>>
>> There would only be a single directory that will contain build files
for
>> all of llvm, clang, lld, etc. The build files would be in
>> llvm/whatever/gn/tree/{llvm,clang,lld,...}/... ("tree" can be
any name).
>> That's not the most natural way to use gn, but I agree that having
BUILD.gn
>> files scattered all over the tree could make it look like gn is
supported
>> at some level.
>>
>> If adding the files in an isolated directory still ends up being
>> confusing in practice, we can move them out at that point. But I'd
like to
>> start with the simpler way and see if that's good enough.
>>
>> On Mon, Nov 5, 2018 at 8:22 PM Justin Bogner <mail at
justinbogner.com>
>> wrote:
>>
>>> Nico Weber via llvm-dev <llvm-dev at lists.llvm.org> writes:
>>> > If I read this correctly, there isn't much opposition to
landing the gn
>>> > files as long as it's very clear that regular devs
aren't supposed to
>>> > update them and that it's clear that they're
experimental
>>> >
>>> > The main concerns I've heard so far:
>>> >
>>> > - Having two build systems is confusing. I can see this, but I
think
>>> > putting the gn files below llvm/experimental/gn (instead of
right into
>>> the
>>> > source, like I currently have) and updating GN.rst to
explicitly say
>>> > "Reviewers should not ask patch authors to update gn
files in addition
>>> to
>>> > the cmake files" will address this.
>>>
>>> I haven't been following this discussion and I don't really
know
>>> anything about GN, but I'm not very convinced by this.
>>>
>>> I'm opposed to adding a second build system in general. We all
remember
>>> having two build systems before and it's a pain keep track of
as well as
>>> a documentation nightmare in telling people yet more ways to do
>>> things. Our docs are already confusing enough given that we
currently
>>> support two completely different layouts of source directories for
>>> cmake.
>>>
>>> I suppose putting it somewhere under utils/ as a kind of tool for
people
>>> who want to use it like we do with text editor support and such
things
>>> wouldn't be the end of the world, but the value of that is
pretty
>>> limited if you're explicitly opting them out of community
maintenance.
>>>
>>> If the build files can be maintained in their own directory, and
will
>>> only be updated by some set of people who are using them, why not
track
>>> them in a separate source repo? This way when a breaking change
happens
>>> your build system could still be made to work for the commits in
between
>>> the time that happens and the GN files are updated, since you'd
be
>>> maintaining a separate mapping of which versions work together.
>>>
>>> > - Having a few people care about the GN build means these
people won't
>>> > improve the cmake build. I think this has some merit (but I
did clean
>>> up
>>> > parts of the cmake build a bit while reading all our cmake
code to
>>> create
>>> > the gn build for example, and I have several ideas about
improving the
>>> > cmake build I want to implement at some point; and it's
also not a
>>> given
>>> > that the folks who may or may not end up working on the GN
build would
>>> have
>>> > worked on the cmake build). However, this is true
independently of
>>> where
>>> > the GN build files are stored.
>>> >
>>> > - GN isn't a cmake replacement, for distro reasons and
whatnot. I
>>> > wholeheartedly agree with this. Maybe GN will become better
here, but
>>> cmake
>>> > is ubiquitous _today_, and it's backed by a company
who's interested in
>>> > keeping it around, so it will be around for a long time. I
think this
>>> is
>>> > cmake's biggest strength. That's why I'm not
proposing on replacing the
>>> > cmake build. If the GN build at some point is way better
(compiler-rt,
>>> > lldb, etc added; out-of-tree build support; GN itself gets a
real
>>> distro
>>> > story; ...) then this _may_ be different in a few years, at
which
>>> point we
>>> > could reconsider. I think this is unlikely to happen.
>>> >
>>> > (I still don't see that any of these problems are being
solved by
>>> having
>>> > this in an overlay or a separate fork.)
>>> >
>>> > And again, It's a real possibility that we check this in
and it turns
>>> out
>>> > the people who are interested in don't feel it's all
that useful, it
>>> > bitrots, and we delete it again. That's cool, no harm
done. I think we
>>> > should have a very high standard for replacing the supported
build
>>> system
>>> > (cmake), but I think it's cool to have a low bar for
people
>>> experimenting
>>> > with unsupported build systems, as long as they get deleted if
not
>>> used. If
>>> > someone wanted to a, say, llvm/experimental/meson to see how
that would
>>> > feel, I think that'd be super cool too for example.
>>> >
>>> > On Fri, Nov 2, 2018 at 2:06 AM Dean Michael Berris <
>>> dean.berris at gmail.com>
>>> > wrote:
>>> >
>>> >> I think this is how, after moving to a monorepo, it may be
feasible to
>>> >> get this in a separate fork.
>>> >>
>>> >> Granted the Git Monorepo + GitHub happens, we can even
make it so that
>>> >> the GN build is a branch on the official git repository,
which can
>>> >> track the mainline development.
>>> >> On Fri, Nov 2, 2018 at 3:49 AM David Blaikie via llvm-dev
>>> >> <llvm-dev at lists.llvm.org> wrote:
>>> >> >
>>> >> > Any easy way to do this as some kind of overlay, so
they GN files
>>> >> wouldn't live in the LLVM repository, but in a
separate one?
>>> >> >
>>> >> > (this might avoid some of the community discussion -
though would
>>> >> perhaps still likely have the issue I see as most
significant: With a
>>> >> sufficient number of developers using GN, the rate of
build breaks
>>> due to
>>> >> those developers missing CMake file updates might rise to
be a bit of
>>> a
>>> >> drag on the LLVM project - though I don't think
that's likely to ever
>>> be a
>>> >> huge deal, just an annoyance)
>>> >> >
>>> >> > On Wed, Oct 31, 2018 at 11:19 AM Nico Weber via
cfe-dev <
>>> >> cfe-dev at lists.llvm.org> wrote:
>>> >> >>
>>> >> >> Hi,
>>> >> >>
>>> >> >> first things first: If you're happy with
cmake, you can stop
>>> reading
>>> >> now. Nobody is proposing that LLVM moves off cmake, and
nobody is
>>> proposing
>>> >> anything that's causing people using cmake more work.
>>> >> >>
>>> >> >> At the LLVM conference, I gave a lightning talk
[1] about using GN
>>> [2]
>>> >> to build LLVM and clang. cmake is great for many use
cases, but in my
>>> >> opinion local day-to-day development isn't one of
them. So I wrote GN
>>> build
>>> >> files for LLVM and clang, enough to make `ninja check-llvm
check-clang
>>> >> check-lld` build everything needed for these three test
suites and
>>> that all
>>> >> tests pass. This works on Linux, Mac, Win hosts targeting
X86, ARM,
>>> >> AArch64. You can see them at [3].
>>> >> >>
>>> >> >> I had been worried that it would be a lot of work
to keep the build
>>> >> files up to date, but I've been using this for all my
LLVM/clang/lld
>>> >> development the last 8 months, and it turned out to not be
a big
>>> problem --
>>> >> LLVM's build files don't change very often, and GN
build files are a
>>> >> pleasure to work with in my opinion.
>>> >> >>
>>> >> >> I gave the lightning talk just to talk about my
personal workflow,
>>> but
>>> >> there was a lot of interest. We had a roundtable on the
next day, and
>>> about
>>> >> 20 people said they'd be interested in using this for
their
>>> development
>>> >> too. The main request was that the .gn files are checked
in upstream,
>>> so
>>> >> that we can collaborate on keeping them working. Two to
three orgs
>>> even
>>> >> said they'd be interested in moving their buildbots to
GN.
>>> >> >>
>>> >> >> As mentioned at the top, the intention here is
not to replace
>>> cmake,
>>> >> only to offer an opt-in alternative for people who are
interested in
>>> it.
>>> >> Keeping the GN build going would be the responsibility of
people
>>> using it,
>>> >> not of the general LLVM community. If this fails to find
use and
>>> bitrots,
>>> >> we can easily remove it again.
>>> >> >>
>>> >> >> Are there any concerns with checking in GN files?
I've put some
>>> initial
>>> >> docs for the GN build at https://reviews.llvm.org/D53944 ;
it
>>> describes
>>> >> what the GN build is and is not, what its advantages are
(speed and
>>> easier
>>> >> configurability), and some points about the philosophy
behind the
>>> LLVM GN
>>> >> build.
>>> >> >>
>>> >> >> If folks are generally ok with GN files living
in-tree, I'll start
>>> >> sending patches for gradually adding gn files through the
regular
>>> review
>>> >> process.
>>> >> >>
>>> >> >> If having a BUILD.gn file in every directory
being confusing is a
>>> >> concern, GN has the concept of a "secondary
tree" so that all GN files
>>> >> could be below llvm/gn/tree/{llvm,clang,lld,...}.
>>> >> >>
>>> >> >> Cheers,
>>> >> >> Nico
>>> >> >>
>>> >> >> 1:
https://llvm.org/devmtg/2018-10/talk-abstracts.html#lt2
>>> >> >> 2: https://gn.googlesource.com/gn ,
https://is.gd/gn_intro
>>> >> >> 3:
>>> >>
>>>
https://github.com/llvm-project/llvm-project-20170507/compare/master...nico:gn
>>> >> , click "Files Changed" to see the GN files.
>>> >> >> _______________________________________________
>>> >> >> cfe-dev mailing list
>>> >> >> cfe-dev at lists.llvm.org
>>> >> >>
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>> >> >
>>> >> > _______________________________________________
>>> >> > LLVM Developers mailing list
>>> >> > llvm-dev at lists.llvm.org
>>> >> >
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Dean
>>> >>
>>> > _______________________________________________
>>> > LLVM Developers mailing list
>>> > llvm-dev at lists.llvm.org
>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181106/ab4cf898/attachment-0001.html>

At one point in lldb someone suggested the name contrib for the Xcode
project files.  That seemed like a good suggestion, but it never
materialized.  And admittedly there is no existing contrib folder in LLVM,
so it may be better to repurpose an existing one than to create a new one.
But if contrib could end up serving other purposes besides housing a set of
gn files, then it's at least a well-understood name that very precisely the
level of support the project guarantees with it.

On Tue, Nov 6, 2018 at 11:55 AM Nico Weber via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> Awesome. I'm happy with moving the .rst file into that directory and
not
> have it on the public website. I'll try to make a patch that lands
enough
> scaffolding to build `not` in the next few days, and then I'll land the
> other build files I have through the regular build process after that.
> Unless someone feels strongly, I'll go with Justin's suggestion of
> llvm/utils/gn/...
>
> On Tue, Nov 6, 2018 at 12:20 PM David Blaikie <dblaikie at gmail.com>
wrote:
>
>> Yeah, I don't feel super strongly about the name or location if
it's a
>> single directory entry point. Not sure if/where it should be documented
(if
>> it needs web documentation - rather than or in addition to a README in
that
>> directory) - don't really want to muddy the waters in the official
docs as
>> others have mentioned. I'm OK with it going in-tree under those
conditions,
>> FWIW.
>>
>> On Tue, Nov 6, 2018 at 3:10 AM Nico Weber via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> The value in having them somewhere in-tree is that it's easier
for
>>> people collaborate on these files, and it's way lower setup
overhead if
>>> someone wants to try it out. If people prefer llvm/util over
>>> llvm/experimental, that's fine with me.
>>>
>>> There would only be a single directory that will contain build
files for
>>> all of llvm, clang, lld, etc. The build files would be in
>>> llvm/whatever/gn/tree/{llvm,clang,lld,...}/... ("tree"
can be any name).
>>> That's not the most natural way to use gn, but I agree that
having BUILD.gn
>>> files scattered all over the tree could make it look like gn is
supported
>>> at some level.
>>>
>>> If adding the files in an isolated directory still ends up being
>>> confusing in practice, we can move them out at that point. But
I'd like to
>>> start with the simpler way and see if that's good enough.
>>>
>>> On Mon, Nov 5, 2018 at 8:22 PM Justin Bogner <mail at
justinbogner.com>
>>> wrote:
>>>
>>>> Nico Weber via llvm-dev <llvm-dev at lists.llvm.org>
writes:
>>>> > If I read this correctly, there isn't much opposition
to landing the
>>>> gn
>>>> > files as long as it's very clear that regular devs
aren't supposed to
>>>> > update them and that it's clear that they're
experimental
>>>> >
>>>> > The main concerns I've heard so far:
>>>> >
>>>> > - Having two build systems is confusing. I can see this,
but I think
>>>> > putting the gn files below llvm/experimental/gn (instead
of right
>>>> into the
>>>> > source, like I currently have) and updating GN.rst to
explicitly say
>>>> > "Reviewers should not ask patch authors to update gn
files in
>>>> addition to
>>>> > the cmake files" will address this.
>>>>
>>>> I haven't been following this discussion and I don't
really know
>>>> anything about GN, but I'm not very convinced by this.
>>>>
>>>> I'm opposed to adding a second build system in general. We
all remember
>>>> having two build systems before and it's a pain keep track
of as well as
>>>> a documentation nightmare in telling people yet more ways to do
>>>> things. Our docs are already confusing enough given that we
currently
>>>> support two completely different layouts of source directories
for
>>>> cmake.
>>>>
>>>> I suppose putting it somewhere under utils/ as a kind of tool
for people
>>>> who want to use it like we do with text editor support and such
things
>>>> wouldn't be the end of the world, but the value of that is
pretty
>>>> limited if you're explicitly opting them out of community
maintenance.
>>>>
>>>> If the build files can be maintained in their own directory,
and will
>>>> only be updated by some set of people who are using them, why
not track
>>>> them in a separate source repo? This way when a breaking change
happens
>>>> your build system could still be made to work for the commits
in between
>>>> the time that happens and the GN files are updated, since
you'd be
>>>> maintaining a separate mapping of which versions work together.
>>>>
>>>> > - Having a few people care about the GN build means these
people won't
>>>> > improve the cmake build. I think this has some merit (but
I did clean
>>>> up
>>>> > parts of the cmake build a bit while reading all our cmake
code to
>>>> create
>>>> > the gn build for example, and I have several ideas about
improving the
>>>> > cmake build I want to implement at some point; and
it's also not a
>>>> given
>>>> > that the folks who may or may not end up working on the GN
build
>>>> would have
>>>> > worked on the cmake build). However, this is true
independently of
>>>> where
>>>> > the GN build files are stored.
>>>> >
>>>> > - GN isn't a cmake replacement, for distro reasons and
whatnot. I
>>>> > wholeheartedly agree with this. Maybe GN will become
better here, but
>>>> cmake
>>>> > is ubiquitous _today_, and it's backed by a company
who's interested
>>>> in
>>>> > keeping it around, so it will be around for a long time. I
think this
>>>> is
>>>> > cmake's biggest strength. That's why I'm not
proposing on replacing
>>>> the
>>>> > cmake build. If the GN build at some point is way better
(compiler-rt,
>>>> > lldb, etc added; out-of-tree build support; GN itself gets
a real
>>>> distro
>>>> > story; ...) then this _may_ be different in a few years,
at which
>>>> point we
>>>> > could reconsider. I think this is unlikely to happen.
>>>> >
>>>> > (I still don't see that any of these problems are
being solved by
>>>> having
>>>> > this in an overlay or a separate fork.)
>>>> >
>>>> > And again, It's a real possibility that we check this
in and it turns
>>>> out
>>>> > the people who are interested in don't feel it's
all that useful, it
>>>> > bitrots, and we delete it again. That's cool, no harm
done. I think we
>>>> > should have a very high standard for replacing the
supported build
>>>> system
>>>> > (cmake), but I think it's cool to have a low bar for
people
>>>> experimenting
>>>> > with unsupported build systems, as long as they get
deleted if not
>>>> used. If
>>>> > someone wanted to a, say, llvm/experimental/meson to see
how that
>>>> would
>>>> > feel, I think that'd be super cool too for example.
>>>> >
>>>> > On Fri, Nov 2, 2018 at 2:06 AM Dean Michael Berris <
>>>> dean.berris at gmail.com>
>>>> > wrote:
>>>> >
>>>> >> I think this is how, after moving to a monorepo, it
may be feasible
>>>> to
>>>> >> get this in a separate fork.
>>>> >>
>>>> >> Granted the Git Monorepo + GitHub happens, we can even
make it so
>>>> that
>>>> >> the GN build is a branch on the official git
repository, which can
>>>> >> track the mainline development.
>>>> >> On Fri, Nov 2, 2018 at 3:49 AM David Blaikie via
llvm-dev
>>>> >> <llvm-dev at lists.llvm.org> wrote:
>>>> >> >
>>>> >> > Any easy way to do this as some kind of overlay,
so they GN files
>>>> >> wouldn't live in the LLVM repository, but in a
separate one?
>>>> >> >
>>>> >> > (this might avoid some of the community
discussion - though would
>>>> >> perhaps still likely have the issue I see as most
significant: With a
>>>> >> sufficient number of developers using GN, the rate of
build breaks
>>>> due to
>>>> >> those developers missing CMake file updates might rise
to be a bit
>>>> of a
>>>> >> drag on the LLVM project - though I don't think
that's likely to
>>>> ever be a
>>>> >> huge deal, just an annoyance)
>>>> >> >
>>>> >> > On Wed, Oct 31, 2018 at 11:19 AM Nico Weber via
cfe-dev <
>>>> >> cfe-dev at lists.llvm.org> wrote:
>>>> >> >>
>>>> >> >> Hi,
>>>> >> >>
>>>> >> >> first things first: If you're happy with
cmake, you can stop
>>>> reading
>>>> >> now. Nobody is proposing that LLVM moves off cmake,
and nobody is
>>>> proposing
>>>> >> anything that's causing people using cmake more
work.
>>>> >> >>
>>>> >> >> At the LLVM conference, I gave a lightning
talk [1] about using
>>>> GN [2]
>>>> >> to build LLVM and clang. cmake is great for many use
cases, but in my
>>>> >> opinion local day-to-day development isn't one of
them. So I wrote
>>>> GN build
>>>> >> files for LLVM and clang, enough to make `ninja
check-llvm
>>>> check-clang
>>>> >> check-lld` build everything needed for these three
test suites and
>>>> that all
>>>> >> tests pass. This works on Linux, Mac, Win hosts
targeting X86, ARM,
>>>> >> AArch64. You can see them at [3].
>>>> >> >>
>>>> >> >> I had been worried that it would be a lot of
work to keep the
>>>> build
>>>> >> files up to date, but I've been using this for all
my LLVM/clang/lld
>>>> >> development the last 8 months, and it turned out to
not be a big
>>>> problem --
>>>> >> LLVM's build files don't change very often,
and GN build files are a
>>>> >> pleasure to work with in my opinion.
>>>> >> >>
>>>> >> >> I gave the lightning talk just to talk about
my personal
>>>> workflow, but
>>>> >> there was a lot of interest. We had a roundtable on
the next day,
>>>> and about
>>>> >> 20 people said they'd be interested in using this
for their
>>>> development
>>>> >> too. The main request was that the .gn files are
checked in
>>>> upstream, so
>>>> >> that we can collaborate on keeping them working. Two
to three orgs
>>>> even
>>>> >> said they'd be interested in moving their
buildbots to GN.
>>>> >> >>
>>>> >> >> As mentioned at the top, the intention here
is not to replace
>>>> cmake,
>>>> >> only to offer an opt-in alternative for people who are
interested in
>>>> it.
>>>> >> Keeping the GN build going would be the responsibility
of people
>>>> using it,
>>>> >> not of the general LLVM community. If this fails to
find use and
>>>> bitrots,
>>>> >> we can easily remove it again.
>>>> >> >>
>>>> >> >> Are there any concerns with checking in GN
files? I've put some
>>>> initial
>>>> >> docs for the GN build at
https://reviews.llvm.org/D53944 ; it
>>>> describes
>>>> >> what the GN build is and is not, what its advantages
are (speed and
>>>> easier
>>>> >> configurability), and some points about the philosophy
behind the
>>>> LLVM GN
>>>> >> build.
>>>> >> >>
>>>> >> >> If folks are generally ok with GN files
living in-tree, I'll start
>>>> >> sending patches for gradually adding gn files through
the regular
>>>> review
>>>> >> process.
>>>> >> >>
>>>> >> >> If having a BUILD.gn file in every directory
being confusing is a
>>>> >> concern, GN has the concept of a "secondary
tree" so that all GN
>>>> files
>>>> >> could be below llvm/gn/tree/{llvm,clang,lld,...}.
>>>> >> >>
>>>> >> >> Cheers,
>>>> >> >> Nico
>>>> >> >>
>>>> >> >> 1:
https://llvm.org/devmtg/2018-10/talk-abstracts.html#lt2
>>>> >> >> 2: https://gn.googlesource.com/gn ,
https://is.gd/gn_intro
>>>> >> >> 3:
>>>> >>
>>>>
https://github.com/llvm-project/llvm-project-20170507/compare/master...nico:gn
>>>> >> , click "Files Changed" to see the GN files.
>>>> >> >>
_______________________________________________
>>>> >> >> cfe-dev mailing list
>>>> >> >> cfe-dev at lists.llvm.org
>>>> >> >>
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>> >> >
>>>> >> > _______________________________________________
>>>> >> > LLVM Developers mailing list
>>>> >> > llvm-dev at lists.llvm.org
>>>> >> >
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Dean
>>>> >>
>>>> > _______________________________________________
>>>> > LLVM Developers mailing list
>>>> > llvm-dev at lists.llvm.org
>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181106/ca71b84a/attachment-0001.html>

> At one point in lldb someone suggested the name contrib for the Xcode
> project files.  That seemed like a good suggestion, but it never
> materialized.  And admittedly there is no existing contrib folder in
> LLVM, so it may be better to repurpose an existing one than to create
> a new one.  But if contrib could end up serving other purposes besides
> housing a set of gn files, then it's at least a well-understood name
> that very precisely the level of support the project guarantees with it.
Seems like a great place to put the editor support files (vim, emacs).
--paulr