llvm dev - Oct 2020 - Contributing Bazel BUILD files similar to gn

Hi all,

tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
side-directory in the monorepo, similar to the gn build.

Some of us have been working on open-source Bazel BUILD files for the LLVM
Project. You may have seen us hanging out in the #build-systems discord
channel. As you may know, Google uses Bazel internally and has maintained a
Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
we've got more and more OSS projects with a Bazel BUILD depending on LLVM
(e.g. IREE <https://github.com/google/iree> and TensorFlow
<https://github.com/tensorflow/tensorflow>). We're also not the only
ones
using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've
borrowed
from TF
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
Each of these projects has to jump through some weird hoops to keep their
version of the Bazel BUILD files in sync with the code, which requires some
fragile combination of scripts and human intervention. Instead, we'd like
to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
We expect to follow the model of the GN build where these will be
maintained by interested contributors rather than expecting the general
community to maintain them.

To facilitate and test this we've been developing a standalone repository
that just has the Bazel BUILD files. It symlinks together the directory
trees on top of a submodule as we would need in the monorepo to to avoid
in-tree BUILD files. The configuration is at
https://github.com/google/llvm-bazel. We now have those in a good place and
think they would be useful upstream.

# Details

## What

Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
<https://github.com/google/llvm-bazel/pull/72>) subprojects, potentially
expanding to others, as needed. Basically everything currently at
https://github.com/google/llvm-bazel.

## Where

In https://github.com/google/llvm-bazel the BUILD files live in a single
directory tree matching the structure of the overall llvm-project
directory. For users, @llvm-project is a single Bazel repository
<https://docs.bazel.build/versions/master/build-ref.html#repositories>
that
includes both LLVM and MLIR subprojects. To maintain this structure, we
would probably want to put a `bazel` directory in the monorepo's utils
directory <https://github.com/llvm/llvm-project/tree/master/utils>, which
currently only contains a directory for arcanist. This is different from
gn, which is under the LLVM subproject's utils directory
<https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We could
similarly put the Bazel BUILD files under llvm/utils/bazel but have them be
for the entire llvm project (the subsets that are supported). This seems
like an odd structure to me, but I know that the CMake build for LLVM
also builds
the other subprojects
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
so maybe this would be preferable.

Alternatively we could split each subproject into a separate Bazel
repository and put the Bazel build files under each subproject. I think
this fragments the configuration of the BUILD without much benefit.

## Configurations

We currently have configurations for Linux GCC and Clang, MacOS GCC and
Clang, and Windows MSVC. Support for other configurations can be added
as-desired, but supporting all possible LLVM build configurations is not
the goal.

## Support

Support would be similar to the gn build. Contributors could optionally
update the Bazel BUILD files as part of their patches, but would be under
no obligation to do so.

## Preserving History

I don't *think* the history of llvm-bazel is interesting enough to try to
merge it into the monorepo and I was planning to submit this as a single
patch, but please let me know if you disagree.

## Benefits to the community

   -

   Projects that depend on LLVM and use the Bazel build system can avoid
   duplicating fragile effort. We'll spend more time contributing to LLVM
   instead :-D
   -

   Bazel is stricter than CMake in many ways (e.g. it requires that even
   header dependencies be declared) and can catch layering issues very easily.
   There's even an optional layering_check feature we could turn on if its
use
   would benefit the community. (though currently the existing problematic
   layering makes it a burden to maintain on our own). Even without that
   additional check, as I've been keeping the Bazel build green, I've
found
   and fixed a number of layering issues in the past couple weeks (e.g.
   https://reviews.llvm.org/rGb49787df9a
   <https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
   and https://reviews.llvm.org/rGc17ae2916c
   <https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).


Here's a patch <https://reviews.llvm.org/D90352> adding the Bazel
build
system. It's basically just `cp -r llvm-bazel/llvm-bazel
llvm-project/utils/bazel`.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201028/89728611/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3992 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201028/89728611/attachment-0001.bin>

On 10/28/20 7:18 PM, Geoffrey Martin-Noble via llvm-dev
wrote:
> Hi all,
> 
> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
> side-directory in the monorepo, similar to the gn build.
> 
> Some of us have been working on open-source Bazel BUILD files for the 
> LLVM Project. You may have seen us hanging out in the #build-systems 
> discord channel. As you may know, Google uses Bazel internally and has 
> maintained a Bazel BUILD of LLVM for years. Especially with the 
> introduction of MLIR, we've got more and more OSS projects with a Bazel
> BUILD depending on LLVM (e.g. IREE
<https://github.com/google/iree>and
> TensorFlow <https://github.com/tensorflow/tensorflow>). We're
also not
> the only ones using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM 
> that they've borrowed from TF 
>
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
> Each of these projects has to jump through some weird hoops to keep 
> their version of the Bazel BUILD files in sync with the code, which 
> requires some fragile combination of scripts and human intervention. 
> Instead, we'd like to move general-purpose Bazel BUILD files into the 
> LLVM Project monorepo. We expect to follow the model of the GN build 
> where these will be maintained by interested contributors rather than 
> expecting the general community to maintain them.
> 
> To facilitate and test this we've been developing a standalone 
> repository that just has the Bazel BUILD files. It symlinks together the 
> directory trees on top of a submodule as we would need in the monorepo 
> to to avoid in-tree BUILD files. The configuration is at 
> https://github.com/google/llvm-bazel. We now have those in a good place 
> and think they would be useful upstream.
> 
Can you explain some of the benefits to using Bazel instead of CMake?

I'm a little concerned about having two 'unsupported' buildsystems 
living in tree, and I'm not sure what would stop us from continuing to 
add more.  I would feel better if we had a set of guidelines to define 
the criteria for adding a new buildsytem and also criteria for when we 
can remove them.

Would you be able to amend this proposal to include some general 
guidelines for adding/removing new buildsystems, so that we can discuss 
that too?

Thanks,
Tom

> 
> # Details
> 
> ## What
> 
> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review 
> <https://github.com/google/llvm-bazel/pull/72>) subprojects,
potentially
> expanding to others, as needed. Basically everything currently at 
> https://github.com/google/llvm-bazel.
> 
> 
> ## Where
> 
> In https://github.com/google/llvm-bazelthe BUILD files live in a single 
> directory tree matching the structure of the overall llvm-project 
> directory. For users, @llvm-project is a single Bazel repository 
>
<https://docs.bazel.build/versions/master/build-ref.html#repositories>that
> includes both LLVM and MLIR subprojects. To maintain this structure, we 
> would probably want to put a `bazel` directory in the monorepo's utils 
> directory <https://github.com/llvm/llvm-project/tree/master/utils>, 
> which currently only contains a directory for arcanist. This is 
> different from gn, which is under the LLVM subproject's utils directory
> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We 
> could similarly put the Bazel BUILD files under llvm/utils/bazel but 
> have them be for the entire llvm project (the subsets that are 
> supported). This seems like an odd structure to me, but I know that the 
> CMake build for LLVM also builds the other subprojects 
>
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
> so maybe this would be preferable.
> 
> Alternatively we could split each subproject into a separate Bazel 
> repository and put the Bazel build files under each subproject. I think 
> this fragments the configuration of the BUILD without much benefit.
> 
> 
> ## Configurations
> 
> We currently have configurations for Linux GCC and Clang, MacOS GCC and 
> Clang, and Windows MSVC. Support for other configurations can be added 
> as-desired, but supporting all possible LLVM build configurations is not 
> the goal.
> 
> 
> ## Support
> 
> Support would be similar to the gn build. Contributors could optionally 
> update the Bazel BUILD files as part of their patches, but would be 
> under no obligation to do so.
> 
> 
> ## Preserving History
> 
> I don't *think* the history of llvm-bazel is interesting enough to try 
> to merge it into the monorepo and I was planning to submit this as a 
> single patch, but please let me know if you disagree.
> 
> 
> ## Benefits to the community
> 
>   *
> 
>     Projects that depend on LLVM and use the Bazel build system can
>     avoid duplicating fragile effort. We'll spend more time
contributing
>     to LLVM instead :-D
> 
>   *
> 
>     Bazel is stricter than CMake in many ways (e.g. it requires that
>     even header dependencies be declared) and can catch layering issues
>     very easily. There's even an optional layering_check feature we
>     could turn on if its use would benefit the community. (though
>     currently the existing problematic layering makes it a burden to
>     maintain on our own). Even without that additional check, as I've
>     been keeping the Bazel build green, I've found and fixed a number
of
>     layering issues in the past couple weeks (e.g.
>     https://reviews.llvm.org/rGb49787df9a
>    
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>and
>     https://reviews.llvm.org/rGc17ae2916c
>    
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
> 
> 
> Here's a patch <https://reviews.llvm.org/D90352>adding the Bazel
build
> system. It's basically just `cp -r llvm-bazel/llvm-bazel 
> llvm-project/utils/bazel`.
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

On Thu, 29 Oct 2020 at 15:23, Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I'm a little concerned about having two 'unsupported'
buildsystems
> living in tree, and I'm not sure what would stop us from continuing to
> add more.  I would feel better if we had a set of guidelines to define
> the criteria for adding a new buildsytem and also criteria for when we
> can remove them.

I have used Bazel and it doesn't seem to map well to CMake. It seems to be
in between CMake and Ninja with a lot of hard-coded dependencies that are
cumbersome to keep updating. I'm by no means an expert, and I could very
well be wrong, but supporting more than one build system is not trivial
(remember the autoconf days?).

For example, when trying to implement the same logic on both will not be
trivial. So, whenever we want to add some functionality or improve how we
build LLVM with one system, we'll have to do so in multiple build systems
that do not easily match each other. If we don't try to match
functionality, we'll segregate the community, because people will be able
to do X on build system A but not B, and the similar features cluster
together and then we have essentially two projects built from the same
source code.

Testing this, or worse, trying to fix a buildbot that is built with Bazel
(and having to install Java JDK and all its dependencies) on potentially a
hardware that you do not have access to, will be a nightmare to debug. The
nature of post-commit testing, revert and review of LLVM will not make that
simpler. Unless we treat the Bazel build as "not our problem" (which
defeats the point of having it?).

To make matters worse, our CMake files are not simple, and do not do all of
the things we want them to do in the way we understand completely. There is
a lot of kludge that we carry and with that comes in two categories: the
things that we hate and would love to fix, and the things that are fixes
that we have no idea are there. The former are the reasons why people want
to start a new build system, the latter is why they soon realise that was a
mistake (insert XKCD joke here).

If the Bazel files can be completely ignored, then it's just more clutter.
But if other projects start to use more different build systems and we
start packing them all in LLVM, then we'll have a hard time knowing what we
build how. I can't really see this scaling.

Two-cents worth.
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/973cd48b/attachment.html>

On Thu, Oct 29, 2020 at 11:23 AM Tom Stellard via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> > tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR
in a
> > side-directory in the monorepo, similar to the gn build.
> Can you explain some of the benefits to using Bazel instead of CMake?
I can, and I will be very brief: None.

-- 
Stefan Teleman
stefan.teleman at gmail.com

(following up on the discussion on Discord)

On Thu, Oct 29, 2020 at 8:23 AM Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 10/28/20 7:18 PM, Geoffrey Martin-Noble via llvm-dev wrote:
> > Hi all,
> >
> > tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR
in a
> > side-directory in the monorepo, similar to the gn build.
> >
> > Some of us have been working on open-source Bazel BUILD files for the
> > LLVM Project. You may have seen us hanging out in the #build-systems
> > discord channel. As you may know, Google uses Bazel internally and has
> > maintained a Bazel BUILD of LLVM for years. Especially with the
> > introduction of MLIR, we've got more and more OSS projects with a
Bazel
> > BUILD depending on LLVM (e.g. IREE
<https://github.com/google/iree>and
> > TensorFlow <https://github.com/tensorflow/tensorflow>).
We're also not
> > the only ones using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM
> > that they've borrowed from TF
> >
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
>
> > Each of these projects has to jump through some weird hoops to keep
> > their version of the Bazel BUILD files in sync with the code, which
> > requires some fragile combination of scripts and human intervention.
> > Instead, we'd like to move general-purpose Bazel BUILD files into
the
> > LLVM Project monorepo. We expect to follow the model of the GN build
> > where these will be maintained by interested contributors rather than
> > expecting the general community to maintain them.
> >
> > To facilitate and test this we've been developing a standalone
> > repository that just has the Bazel BUILD files. It symlinks together
the
> > directory trees on top of a submodule as we would need in the monorepo
> > to to avoid in-tree BUILD files. The configuration is at
> > https://github.com/google/llvm-bazel. We now have those in a good
place
> > and think they would be useful upstream.
> >
>
> Can you explain some of the benefits to using Bazel instead of CMake?
>
The question can be taken with multiple angles:

1) The benefit for the LLVM project to switch from CMake to Bazel: it isn't
clear even that it would be practical/feasible or serve all the users, this
is *not* *the proposal here*.* Nothing changes for CMake.*
2) The benefit for an LLVM developer to use Bazel instead of CMake: I think
it is minor, unless you have access to a remote build farm in your
environment maybe?
3) The benefit for a project that is willing to use LLVM to use Bazel
instead of CMake: this is a tricky topic because there are many variables.
The Bazel website is a good starting point I think: https://bazel.build
Some of the practical thing I perceive as interesting with Bazel over CMake
personally:
  - Strict checking of the dependencies: this seems strange at first that a
missing library dependency won't allow you to include a header which is
present on the filesystem, but this is actually very powerful: this is what
allow Bazel to have correctness with incremental build and what enables
caching at all. This is also quite fundamental to the "distributed
build"
mode in Bazel: you can farm-out the build to a large distributed cluster
with remote cachine.
  - Declarative approach: this enables static analysis of the BUILD
configuration/graphs, and transformations as well. For example because of
the above, the tooling can figure out unused library dependencies, or
adding missing dependencies as well.
These two aspects are impossible to achieve in a principled way in CMake.
Note that it does not imply that I would (or wouldn't) pick Bazel from my
next open-source project, or that I would recommend LLVM to adopt it as a
primary build system!

I can also miss some reasons why some other projects are using Bazel, but
they got >400 attendees at the Bazel conference last year apparently:
https://blog.bazel.build/2019/12/20/bazelcon-2019.html
Ultimately I don't judge why other projects are picking Bazel, we're
just
proposing making their life easier if they start being interested to
include some of LLVM in their project.
In particular, putting on my MLIR ecosystem hat, it'd be great if other
projects using Bazel out-there who may have a possible use for MLIR and
LLVM could have an easier integration path: right now in practice they may
use the Bazel configuration that are shipped inside TensorFlow and try to
adjust them to BUILD LLVM and send us patches to integrate in TensorFlow.
This is difficult because they may want to improve support for the Bazel
config on platforms that TensorFlow does not support: it is just not the
right place for people interested in building with Bazel to collaborate.


I'm a little concerned about having two 'unsupported'
buildsystems
> living in tree, and I'm not sure what would stop us from continuing to
> add more.

Following up on some concerns from Tom (and others) explained on Discord
here:

a) Commit Mailing list traffic for updating these build files. This is a
valid point with the GN bot today. It also shows up when I `git log llvm/`
in the monorepo annoyingly.
The proposal would be to have the Bazel and `gn` files in a separate folder
at the top level of the monorepo: that way no commits email would be sent
to any mailing list, and no update would show up in the `git log llvm/`.
b) CI systems picking up more commits to build when not needed: similarly
as above, isolating these in a separate part of the tree will exclude these
from bots tracking the llvm/ or clang/ paths.
c) Investing in developing Bazel support means less investment in CMake. It
is true that engineers fixing Bazel configs are spending time there instead
of in CMake, however the situation is that downstream projects that picked
Bazel (for their own reasons, I don't judge) who start using LLVM are
spending the time to maintain these Bazel files out-of-tree. We're not
making the situation worse by allowing the maintainer of these projects
(who are also frequently upstream contributors) to just collaborate on
their set of patches in a more coordinated way upstream. Also no public
LLVM bots builds with an unsupported build system, any feature is expected
to build with CMake on every supported platform I believe. It seems like
this worked out well with `gn` in practice?


> I would feel better if we had a set of guidelines to define
> the criteria for adding a new buildsytem and also criteria for when we
> can remove them.
>
> Would you be able to amend this proposal to include some general
> guidelines for adding/removing new buildsystems, so that we can discuss
> that too?
>
>
That's an excellent point as well! Could we take inspiration from the
experimental (or non-experimental) backends? For example if `gn` does not
build anymore, send an email to LLVM-dev@ proposing to remove it and see if
any maintainer steps up? Without a community to maintain it we should
remove these easily and quickly?

-- 
Mehdi






> Thanks,
> Tom
>
>
> >
> > # Details
> >
> > ## What
> >
> > Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
> > <https://github.com/google/llvm-bazel/pull/72>) subprojects,
> potentially
> > expanding to others, as needed. Basically everything currently at
> > https://github.com/google/llvm-bazel.
> >
> >
> > ## Where
> >
> > In https://github.com/google/llvm-bazelthe BUILD files live in a
single
> > directory tree matching the structure of the overall llvm-project
> > directory. For users, @llvm-project is a single Bazel repository
> >
<https://docs.bazel.build/versions/master/build-ref.html#repositories>that
>
> > includes both LLVM and MLIR subprojects. To maintain this structure,
we
> > would probably want to put a `bazel` directory in the monorepo's
utils
> > directory
<https://github.com/llvm/llvm-project/tree/master/utils>,
> > which currently only contains a directory for arcanist. This is
> > different from gn, which is under the LLVM subproject's utils
directory
> >
<https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
> > could similarly put the Bazel BUILD files under llvm/utils/bazel but
> > have them be for the entire llvm project (the subsets that are
> > supported). This seems like an odd structure to me, but I know that
the
> > CMake build for LLVM also builds the other subprojects
> > <
>
https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
>
> > so maybe this would be preferable.
> >
> > Alternatively we could split each subproject into a separate Bazel
> > repository and put the Bazel build files under each subproject. I
think
> > this fragments the configuration of the BUILD without much benefit.
> >
> >
> > ## Configurations
> >
> > We currently have configurations for Linux GCC and Clang, MacOS GCC
and
> > Clang, and Windows MSVC. Support for other configurations can be added
> > as-desired, but supporting all possible LLVM build configurations is
not
> > the goal.
> >
> >
> > ## Support
> >
> > Support would be similar to the gn build. Contributors could
optionally
> > update the Bazel BUILD files as part of their patches, but would be
> > under no obligation to do so.
> >
> >
> > ## Preserving History
> >
> > I don't *think* the history of llvm-bazel is interesting enough to
try
> > to merge it into the monorepo and I was planning to submit this as a
> > single patch, but please let me know if you disagree.
> >
> >
> > ## Benefits to the community
> >
> >   *
> >
> >     Projects that depend on LLVM and use the Bazel build system can
> >     avoid duplicating fragile effort. We'll spend more time
contributing
> >     to LLVM instead :-D
> >
> >   *
> >
> >     Bazel is stricter than CMake in many ways (e.g. it requires that
> >     even header dependencies be declared) and can catch layering
issues
> >     very easily. There's even an optional layering_check feature
we
> >     could turn on if its use would benefit the community. (though
> >     currently the existing problematic layering makes it a burden to
> >     maintain on our own). Even without that additional check, as
I've
> >     been keeping the Bazel build green, I've found and fixed a
number of
> >     layering issues in the past couple weeks (e.g.
> >     https://reviews.llvm.org/rGb49787df9a
> >    
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d
> >and
> >     https://reviews.llvm.org/rGc17ae2916c
> >    
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a
> >).
> >
> >
> > Here's a patch <https://reviews.llvm.org/D90352>adding the
Bazel build
> > system. It's basically just `cp -r llvm-bazel/llvm-bazel
> > llvm-project/utils/bazel`.
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/9dc357bd/attachment.html>

I want to jump in with some general support for this addition as a
non-googler + frequent bazel user. I find moving between projects that use
bazel much more palatable than moving between bazel + cmake projects.

I also think there's a huge benefit in having strict dependencies. In my
experience this is especially true for tests. This way you know you've
always correctly rebuilt the necessary inputs to a single test, vs using
`lit` directly and having to know / remember which set of binaries are
required to be rebuilt based on your current changes.
--
Keith Smiley


On Wed, Oct 28, 2020 at 4:18 PM Geoffrey Martin-Noble via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi all,
>
> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
> side-directory in the monorepo, similar to the gn build.
>
> Some of us have been working on open-source Bazel BUILD files for the LLVM
> Project. You may have seen us hanging out in the #build-systems discord
> channel. As you may know, Google uses Bazel internally and has maintained a
> Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
> we've got more and more OSS projects with a Bazel BUILD depending on
LLVM
> (e.g. IREE <https://github.com/google/iree> and TensorFlow
> <https://github.com/tensorflow/tensorflow>). We're also not the
only ones
> using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've
borrowed
> from TF
>
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
> Each of these projects has to jump through some weird hoops to keep their
> version of the Bazel BUILD files in sync with the code, which requires some
> fragile combination of scripts and human intervention. Instead, we'd
like
> to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
> We expect to follow the model of the GN build where these will be
> maintained by interested contributors rather than expecting the general
> community to maintain them.
>
> To facilitate and test this we've been developing a standalone
repository
> that just has the Bazel BUILD files. It symlinks together the directory
> trees on top of a submodule as we would need in the monorepo to to avoid
> in-tree BUILD files. The configuration is at
> https://github.com/google/llvm-bazel. We now have those in a good place
> and think they would be useful upstream.
>
> # Details
>
> ## What
>
> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
> <https://github.com/google/llvm-bazel/pull/72>) subprojects,
potentially
> expanding to others, as needed. Basically everything currently at
> https://github.com/google/llvm-bazel.
>
> ## Where
>
> In https://github.com/google/llvm-bazel the BUILD files live in a single
> directory tree matching the structure of the overall llvm-project
> directory. For users, @llvm-project is a single Bazel repository
>
<https://docs.bazel.build/versions/master/build-ref.html#repositories>
> that includes both LLVM and MLIR subprojects. To maintain this structure,
> we would probably want to put a `bazel` directory in the monorepo's
utils
> directory <https://github.com/llvm/llvm-project/tree/master/utils>,
which
> currently only contains a directory for arcanist. This is different from
> gn, which is under the LLVM subproject's utils directory
> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
> could similarly put the Bazel BUILD files under llvm/utils/bazel but have
> them be for the entire llvm project (the subsets that are supported). This
> seems like an odd structure to me, but I know that the CMake build for LLVM
> also builds the other subprojects
>
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
> so maybe this would be preferable.
>
> Alternatively we could split each subproject into a separate Bazel
> repository and put the Bazel build files under each subproject. I think
> this fragments the configuration of the BUILD without much benefit.
>
> ## Configurations
>
> We currently have configurations for Linux GCC and Clang, MacOS GCC and
> Clang, and Windows MSVC. Support for other configurations can be added
> as-desired, but supporting all possible LLVM build configurations is not
> the goal.
>
> ## Support
>
> Support would be similar to the gn build. Contributors could optionally
> update the Bazel BUILD files as part of their patches, but would be under
> no obligation to do so.
>
> ## Preserving History
>
> I don't *think* the history of llvm-bazel is interesting enough to try
to
> merge it into the monorepo and I was planning to submit this as a single
> patch, but please let me know if you disagree.
>
> ## Benefits to the community
>
>    -
>
>    Projects that depend on LLVM and use the Bazel build system can avoid
>    duplicating fragile effort. We'll spend more time contributing to
LLVM
>    instead :-D
>    -
>
>    Bazel is stricter than CMake in many ways (e.g. it requires that even
>    header dependencies be declared) and can catch layering issues very
easily.
>    There's even an optional layering_check feature we could turn on if
its use
>    would benefit the community. (though currently the existing problematic
>    layering makes it a burden to maintain on our own). Even without that
>    additional check, as I've been keeping the Bazel build green,
I've found
>    and fixed a number of layering issues in the past couple weeks (e.g.
>    https://reviews.llvm.org/rGb49787df9a
>   
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
>    and https://reviews.llvm.org/rGc17ae2916c
>   
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
>
>
> Here's a patch <https://reviews.llvm.org/D90352> adding the Bazel
build
> system. It's basically just `cp -r llvm-bazel/llvm-bazel
> llvm-project/utils/bazel`.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/4e3a06be/attachment.html>

The main benefit I see is the ease of integrating into an existing Bazel build
system.

At Facebook, we use Buck (which is inspired by Blaze, as is Bazel). Our main
development repository uses Buck (and you get a variety of benefits from such
build systems when you have the required infrastructure integration: remote
caching, distributed builds, hermeticity, etc.), and the Buck build sets up
particular flags, uses a specific sysroot, etc. We have people who want to
develop LLVM-based tooling (that uses the LLVM and Clang libraries) in this
repository, which means the LLVM and Clang libraries we build with CMake also
need to be built with the same flags, sysroot, etc., which entails a bunch of
duplication (and keeping up with any changes to the Buck build system). It's
much more convenient to be able to build the LLVM libraries directly with the
build system you're using for the rest of your build, so that they
automatically get the right build settings.

We build our libraries internally with CMake today, but we've considered
moving them to Buck for this reason. Having Bazel files in-tree would be mildly
more convenient for us (Buck and Bazel are similar enough that we think we could
machine-translate the Bazel files for our use), although we're also fine
grabbing them from some other public repository.

On 10/29/20, 8:23 AM, "llvm-dev on behalf of Tom Stellard via
llvm-dev" <llvm-dev-bounces at lists.llvm.org on behalf of llvm-dev at
lists.llvm.org> wrote:

    On 10/28/20 7:18 PM, Geoffrey Martin-Noble via llvm-dev wrote:
    > Hi all,
    > 
    > tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR
in a
    > side-directory in the monorepo, similar to the gn build.
    > 
    > Some of us have been working on open-source Bazel BUILD files for the 
    > LLVM Project. You may have seen us hanging out in the #build-systems 
    > discord channel. As you may know, Google uses Bazel internally and has 
    > maintained a Bazel BUILD of LLVM for years. Especially with the 
    > introduction of MLIR, we've got more and more OSS projects with a
Bazel
    > BUILD depending on LLVM (e.g. IREE
<https://github.com/google/iree>and
    > TensorFlow <https://github.com/tensorflow/tensorflow>). We're
also not
    > the only ones using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM 
    > that they've borrowed from TF 
    >
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
    > Each of these projects has to jump through some weird hoops to keep 
    > their version of the Bazel BUILD files in sync with the code, which 
    > requires some fragile combination of scripts and human intervention. 
    > Instead, we'd like to move general-purpose Bazel BUILD files into
the
    > LLVM Project monorepo. We expect to follow the model of the GN build 
    > where these will be maintained by interested contributors rather than 
    > expecting the general community to maintain them.
    > 
    > To facilitate and test this we've been developing a standalone 
    > repository that just has the Bazel BUILD files. It symlinks together
the
    > directory trees on top of a submodule as we would need in the monorepo 
    > to to avoid in-tree BUILD files. The configuration is at 
    > https://github.com/google/llvm-bazel. We now have those in a good place
    > and think they would be useful upstream.
    > 

    Can you explain some of the benefits to using Bazel instead of CMake?

    I'm a little concerned about having two 'unsupported'
buildsystems
    living in tree, and I'm not sure what would stop us from continuing to 
    add more.  I would feel better if we had a set of guidelines to define 
    the criteria for adding a new buildsytem and also criteria for when we 
    can remove them.

    Would you be able to amend this proposal to include some general 
    guidelines for adding/removing new buildsystems, so that we can discuss 
    that too?

    Thanks,
    Tom


    > 
    > # Details
    > 
    > ## What
    > 
    > Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review 
    > <https://github.com/google/llvm-bazel/pull/72>) subprojects,
potentially
    > expanding to others, as needed. Basically everything currently at 
    > https://github.com/google/llvm-bazel.
    > 
    > 
    > ## Where
    > 
    > In https://github.com/google/llvm-bazelthe BUILD files live in a single
    > directory tree matching the structure of the overall llvm-project 
    > directory. For users, @llvm-project is a single Bazel repository 
    >
<https://docs.bazel.build/versions/master/build-ref.html#repositories
>that
    > includes both LLVM and MLIR subprojects. To maintain this structure, we
    > would probably want to put a `bazel` directory in the monorepo's
utils
    > directory
<https://github.com/llvm/llvm-project/tree/master/utils>,
    > which currently only contains a directory for arcanist. This is 
    > different from gn, which is under the LLVM subproject's utils
directory
    > <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>.
We
    > could similarly put the Bazel BUILD files under llvm/utils/bazel but 
    > have them be for the entire llvm project (the subsets that are 
    > supported). This seems like an odd structure to me, but I know that the
    > CMake build for LLVM also builds the other subprojects 
    >
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
    > so maybe this would be preferable.
    > 
    > Alternatively we could split each subproject into a separate Bazel 
    > repository and put the Bazel build files under each subproject. I think
    > this fragments the configuration of the BUILD without much benefit.
    > 
    > 
    > ## Configurations
    > 
    > We currently have configurations for Linux GCC and Clang, MacOS GCC and
    > Clang, and Windows MSVC. Support for other configurations can be added 
    > as-desired, but supporting all possible LLVM build configurations is
not
    > the goal.
    > 
    > 
    > ## Support
    > 
    > Support would be similar to the gn build. Contributors could optionally
    > update the Bazel BUILD files as part of their patches, but would be 
    > under no obligation to do so.
    > 
    > 
    > ## Preserving History
    > 
    > I don't *think* the history of llvm-bazel is interesting enough to
try
    > to merge it into the monorepo and I was planning to submit this as a 
    > single patch, but please let me know if you disagree.
    > 
    > 
    > ## Benefits to the community
    > 
    >   *
    > 
    >     Projects that depend on LLVM and use the Bazel build system can
    >     avoid duplicating fragile effort. We'll spend more time
contributing
    >     to LLVM instead :-D
    > 
    >   *
    > 
    >     Bazel is stricter than CMake in many ways (e.g. it requires that
    >     even header dependencies be declared) and can catch layering issues
    >     very easily. There's even an optional layering_check feature we
    >     could turn on if its use would benefit the community. (though
    >     currently the existing problematic layering makes it a burden to
    >     maintain on our own). Even without that additional check, as
I've
    >     been keeping the Bazel build green, I've found and fixed a
number of
    >     layering issues in the past couple weeks (e.g.
    >     https://reviews.llvm.org/rGb49787df9a 
    >    
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d >and
    >     https://reviews.llvm.org/rGc17ae2916c 
    >    
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a >).
    > 
    > 
    > Here's a patch <https://reviews.llvm.org/D90352 >adding the
Bazel build
    > system. It's basically just `cp -r llvm-bazel/llvm-bazel 
    > llvm-project/utils/bazel`.
    > 
    > _______________________________________________
    > LLVM Developers mailing list
    > llvm-dev at lists.llvm.org
    > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev 
    > 

    _______________________________________________
    LLVM Developers mailing list
    llvm-dev at lists.llvm.org
    https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

(see below)


On 10/28/20 6:18 PM, Geoffrey Martin-Noble via llvm-dev wrote:
 > Hi all,
 >
 > tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in
a
 > side-directory in the monorepo, similar to the gn build.
 >
 > Some of us have been working on open-source Bazel BUILD files for the 
LLVM
 > Project. You may have seen us hanging out in the #build-systems discord
 > channel. As you may know, Google uses Bazel internally and has 
maintained a
 > Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
 > we've got more and more OSS projects with a Bazel BUILD depending on
LLVM
 > (e.g. IREE <https://github.com/google/iree> and TensorFlow
 > <https://github.com/tensorflow/tensorflow>). We're also not the
only ones
 > using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've 
borrowed
 > from TF
 >
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
 > Each of these projects has to jump through some weird hoops to keep their
 > version of the Bazel BUILD files in sync with the code, which 
requires some
 > fragile combination of scripts and human intervention. Instead, we'd
like
 > to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
 > We expect to follow the model of the GN build where these will be
 > maintained by interested contributors rather than expecting the general
 > community to maintain them.
 >
 > To facilitate and test this we've been developing a standalone
repository
 > that just has the Bazel BUILD files. It symlinks together the directory
 > trees on top of a submodule as we would need in the monorepo to to avoid
 > in-tree BUILD files. The configuration is at
 > https://github.com/google/llvm-bazel. We now have those in a good 
place and
 > think they would be useful upstream.
 >
 > # Details
 >
 > ## What
 >
 > Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
 > <https://github.com/google/llvm-bazel/pull/72>) subprojects,
potentially
 > expanding to others, as needed. Basically everything currently at
 > https://github.com/google/llvm-bazel.
 >
 > ## Where
 >
 > In https://github.com/google/llvm-bazel the BUILD files live in a single
 > directory tree matching the structure of the overall llvm-project
 > directory. For users, @llvm-project is a single Bazel repository
 > 
<https://docs.bazel.build/versions/master/build-ref.html#repositories>
that
 > includes both LLVM and MLIR subprojects. To maintain this structure, we
 > would probably want to put a `bazel` directory in the monorepo's utils
 > directory <https://github.com/llvm/llvm-project/tree/master/utils>,
which
 > currently only contains a directory for arcanist. This is different from
 > gn, which is under the LLVM subproject's utils directory
 > <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
could
 > similarly put the Bazel BUILD files under llvm/utils/bazel but have 
them be
 > for the entire llvm project (the subsets that are supported). This seems
 > like an odd structure to me, but I know that the CMake build for LLVM
 > also builds
 > the other subprojects
 > 
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
 > so maybe this would be preferable.
 >
 > Alternatively we could split each subproject into a separate Bazel
 > repository and put the Bazel build files under each subproject. I think
 > this fragments the configuration of the BUILD without much benefit.
 >
 > ## Configurations
 >
 > We currently have configurations for Linux GCC and Clang, MacOS GCC and
 > Clang, and Windows MSVC. Support for other configurations can be added
 > as-desired, but supporting all possible LLVM build configurations is not
 > the goal.
 >
 > ## Support
 >
 > Support would be similar to the gn build. Contributors could optionally
 > update the Bazel BUILD files as part of their patches, but would be under
 > no obligation to do so.
 >
 > ## Preserving History
 >
 > I don't *think* the history of llvm-bazel is interesting enough to try
to
 > merge it into the monorepo and I was planning to submit this as a single
 > patch, but please let me know if you disagree.
 >
 > ## Benefits to the community
 >
 >    -
 >
 >    Projects that depend on LLVM and use the Bazel build system can avoid
 >    duplicating fragile effort. We'll spend more time contributing to
LLVM
 >    instead :-D
 >    -
 >
 >    Bazel is stricter than CMake in many ways (e.g. it requires that even
 >    header dependencies be declared) and can catch layering issues 
very easily.
 >    There's even an optional layering_check feature we could turn on 
if its use
 >    would benefit the community. (though currently the existing 
problematic
 >    layering makes it a burden to maintain on our own). Even without that
 >    additional check, as I've been keeping the Bazel build green,
I've
found
 >    and fixed a number of layering issues in the past couple weeks (e.g.
 >    https://reviews.llvm.org/rGb49787df9a
 >
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
 >    and https://reviews.llvm.org/rGc17ae2916c
 >
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
 >
 >
 > Here's a patch <https://reviews.llvm.org/D90352> adding the
Bazel build
 > system. It's basically just `cp -r llvm-bazel/llvm-bazel
 > llvm-project/utils/bazel`.

Doesn't the last paragraph mean all benefits derived from this can be
described either as:
   (1) users do not need to clone the llvm-bazel git repo but get the
       files in llvm-project, or
   (2) "interested contributors" could send patches to llvm-project
       instead of llvm-bazel to update the bazel build.

TBH, I have no interest in using bazel nor anything against it being
merged per se. I just find it curious that we merge another build system
"at no cost" for the community (I think I picked that up in the thread
but I might have imagined the phrasing). I mean, there is always "a
cost"* so it boils down to determine if the benefit is worth it.

~ Johannes


* i.a., people will assume we (=the LLVM community) maintain(s) a bazel
   build, which can certainly be a benefit but also a cost", e.g., when
   the build is not properly maintained, support is scarce, etc. and
   emails come in complaining about it (not thinking of prior examples
   here.)



 > _______________________________________________
 > LLVM Developers mailing list
 > llvm-dev at lists.llvm.org
 > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Thu, Oct 29, 2020 at 9:44 PM Johannes Doerfert via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> (see below)
>
>
> On 10/28/20 6:18 PM, Geoffrey Martin-Noble via llvm-dev wrote:
>  > Hi all,
>  >
>  > tl;dr: We'd like to contribute Bazel BUILD files for LLVM and
MLIR in a
>  > side-directory in the monorepo, similar to the gn build.
>  >
>  > Some of us have been working on open-source Bazel BUILD files for the
> LLVM
>  > Project. You may have seen us hanging out in the #build-systems
discord
>  > channel. As you may know, Google uses Bazel internally and has
> maintained a
>  > Bazel BUILD of LLVM for years. Especially with the introduction of
MLIR,
>  > we've got more and more OSS projects with a Bazel BUILD depending
on
> LLVM
>  > (e.g. IREE <https://github.com/google/iree> and TensorFlow
>  > <https://github.com/tensorflow/tensorflow>). We're also not
the only
> ones
>  > using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that
they've
> borrowed
>  > from TF
>  >
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD
> >.
>  > Each of these projects has to jump through some weird hoops to keep
> their
>  > version of the Bazel BUILD files in sync with the code, which
> requires some
>  > fragile combination of scripts and human intervention. Instead,
we'd
> like
>  > to move general-purpose Bazel BUILD files into the LLVM Project
> monorepo.
>  > We expect to follow the model of the GN build where these will be
>  > maintained by interested contributors rather than expecting the
general
>  > community to maintain them.
>  >
>  > To facilitate and test this we've been developing a standalone
> repository
>  > that just has the Bazel BUILD files. It symlinks together the
directory
>  > trees on top of a submodule as we would need in the monorepo to to
avoid
>  > in-tree BUILD files. The configuration is at
>  > https://github.com/google/llvm-bazel. We now have those in a good
> place and
>  > think they would be useful upstream.
>  >
>  > # Details
>  >
>  > ## What
>  >
>  > Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
>  > <https://github.com/google/llvm-bazel/pull/72>) subprojects,
> potentially
>  > expanding to others, as needed. Basically everything currently at
>  > https://github.com/google/llvm-bazel.
>  >
>  > ## Where
>  >
>  > In https://github.com/google/llvm-bazel the BUILD files live in a
> single
>  > directory tree matching the structure of the overall llvm-project
>  > directory. For users, @llvm-project is a single Bazel repository
>  >
>
<https://docs.bazel.build/versions/master/build-ref.html#repositories>
> that
>  > includes both LLVM and MLIR subprojects. To maintain this structure,
we
>  > would probably want to put a `bazel` directory in the monorepo's
utils
>  > directory
<https://github.com/llvm/llvm-project/tree/master/utils>,
> which
>  > currently only contains a directory for arcanist. This is different
from
>  > gn, which is under the LLVM subproject's utils directory
>  >
<https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
> could
>  > similarly put the Bazel BUILD files under llvm/utils/bazel but have
> them be
>  > for the entire llvm project (the subsets that are supported). This
seems
>  > like an odd structure to me, but I know that the CMake build for LLVM
>  > also builds
>  > the other subprojects
>  >
> <
>
https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41
> >,
>  > so maybe this would be preferable.
>  >
>  > Alternatively we could split each subproject into a separate Bazel
>  > repository and put the Bazel build files under each subproject. I
think
>  > this fragments the configuration of the BUILD without much benefit.
>  >
>  > ## Configurations
>  >
>  > We currently have configurations for Linux GCC and Clang, MacOS GCC
and
>  > Clang, and Windows MSVC. Support for other configurations can be
added
>  > as-desired, but supporting all possible LLVM build configurations is
not
>  > the goal.
>  >
>  > ## Support
>  >
>  > Support would be similar to the gn build. Contributors could
optionally
>  > update the Bazel BUILD files as part of their patches, but would be
> under
>  > no obligation to do so.
>  >
>  > ## Preserving History
>  >
>  > I don't *think* the history of llvm-bazel is interesting enough
to try
> to
>  > merge it into the monorepo and I was planning to submit this as a
single
>  > patch, but please let me know if you disagree.
>  >
>  > ## Benefits to the community
>  >
>  >    -
>  >
>  >    Projects that depend on LLVM and use the Bazel build system can
avoid
>  >    duplicating fragile effort. We'll spend more time contributing
to
> LLVM
>  >    instead :-D
>  >    -
>  >
>  >    Bazel is stricter than CMake in many ways (e.g. it requires that
even
>  >    header dependencies be declared) and can catch layering issues
> very easily.
>  >    There's even an optional layering_check feature we could turn
on
> if its use
>  >    would benefit the community. (though currently the existing
> problematic
>  >    layering makes it a burden to maintain on our own). Even without
that
>  >    additional check, as I've been keeping the Bazel build green,
I've
> found
>  >    and fixed a number of layering issues in the past couple weeks
(e.g.
>  >    https://reviews.llvm.org/rGb49787df9a
>  >
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
>  >    and https://reviews.llvm.org/rGc17ae2916c
>  >
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
>  >
>  >
>  > Here's a patch <https://reviews.llvm.org/D90352> adding the
Bazel build
>  > system. It's basically just `cp -r llvm-bazel/llvm-bazel
>  > llvm-project/utils/bazel`.
>
> Doesn't the last paragraph mean all benefits derived from this can be
> described either as:
>    (1) users do not need to clone the llvm-bazel git repo but get the
>        files in llvm-project, or
>    (2) "interested contributors" could send patches to
llvm-project
>        instead of llvm-bazel to update the bazel build.
>
>
Absolutely. This could happen. The main reason behind this is to make
integating among a number of llvm based projects that use bazel (TF and
TF-based projects primarily, though it sounds like FB's internal process
would be helped as their system is similar to bazel).

> TBH, I have no interest in using bazel nor anything against it being
> merged per se. I just find it curious that we merge another build system
> "at no cost" for the community (I think I picked that up in the
thread
> but I might have imagined the phrasing). I mean, there is always "a
> cost"* so it boils down to determine if the benefit is worth it.
>
>
As far as I can think the cost is...

>
> * i.a., people will assume we (=the LLVM community) maintain(s) a bazel
>    build, which can certainly be a benefit but also a cost", e.g.,
when
>    the build is not properly maintained, support is scarce, etc. and
>    emails come in complaining about it (not thinking of prior examples
>    here.)
>
>
... this. If the system becomes a source of problems or user complaints
then I think it's absolutely reasonable to remove it.

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/eaec3192/attachment.html>

On 10/28/20 7:18 PM, Geoffrey Martin-Noble via llvm-dev
wrote:
> Hi all,
> 
> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
> side-directory in the monorepo, similar to the gn build.
> 
Hi Geoffrey,

I think you've received some good feedback on this thread, but It also 
doesn't look like continuing discussion on this thread is going to lead 
to a consensus.  If you are still interested in making this change, I 
think it would be best to escalate this proposal into a "proposal
pitch"
described here[1].

There is also a template here[2] you can use.

[1] 
https://github.com/llvm/llvm-www/blob/master/proposals/LP0001-LLVMDecisionMaking.md
[2] 
https://github.com/llvm/llvm-www/blob/master/proposals/LP0000-Template.md

-Tom
> Some of us have been working on open-source Bazel BUILD files for the 
> LLVM Project. You may have seen us hanging out in the #build-systems 
> discord channel. As you may know, Google uses Bazel internally and has 
> maintained a Bazel BUILD of LLVM for years. Especially with the 
> introduction of MLIR, we've got more and more OSS projects with a Bazel
> BUILD depending on LLVM (e.g. IREE
<https://github.com/google/iree>and
> TensorFlow <https://github.com/tensorflow/tensorflow>). We're
also not
> the only ones using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM 
> that they've borrowed from TF 
>
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
> Each of these projects has to jump through some weird hoops to keep 
> their version of the Bazel BUILD files in sync with the code, which 
> requires some fragile combination of scripts and human intervention. 
> Instead, we'd like to move general-purpose Bazel BUILD files into the 
> LLVM Project monorepo. We expect to follow the model of the GN build 
> where these will be maintained by interested contributors rather than 
> expecting the general community to maintain them.
> 
> To facilitate and test this we've been developing a standalone 
> repository that just has the Bazel BUILD files. It symlinks together the 
> directory trees on top of a submodule as we would need in the monorepo 
> to to avoid in-tree BUILD files. The configuration is at 
> https://github.com/google/llvm-bazel. We now have those in a good place 
> and think they would be useful upstream.
> 
> 
> # Details
> 
> ## What
> 
> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review 
> <https://github.com/google/llvm-bazel/pull/72>) subprojects,
potentially
> expanding to others, as needed. Basically everything currently at 
> https://github.com/google/llvm-bazel.
> 
> 
> ## Where
> 
> In https://github.com/google/llvm-bazelthe BUILD files live in a single 
> directory tree matching the structure of the overall llvm-project 
> directory. For users, @llvm-project is a single Bazel repository 
>
<https://docs.bazel.build/versions/master/build-ref.html#repositories>that
> includes both LLVM and MLIR subprojects. To maintain this structure, we 
> would probably want to put a `bazel` directory in the monorepo's utils 
> directory <https://github.com/llvm/llvm-project/tree/master/utils>, 
> which currently only contains a directory for arcanist. This is 
> different from gn, which is under the LLVM subproject's utils directory
> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We 
> could similarly put the Bazel BUILD files under llvm/utils/bazel but 
> have them be for the entire llvm project (the subsets that are 
> supported). This seems like an odd structure to me, but I know that the 
> CMake build for LLVM also builds the other subprojects 
>
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
> so maybe this would be preferable.
> 
> Alternatively we could split each subproject into a separate Bazel 
> repository and put the Bazel build files under each subproject. I think 
> this fragments the configuration of the BUILD without much benefit.
> 
> 
> ## Configurations
> 
> We currently have configurations for Linux GCC and Clang, MacOS GCC and 
> Clang, and Windows MSVC. Support for other configurations can be added 
> as-desired, but supporting all possible LLVM build configurations is not 
> the goal.
> 
> 
> ## Support
> 
> Support would be similar to the gn build. Contributors could optionally 
> update the Bazel BUILD files as part of their patches, but would be 
> under no obligation to do so.
> 
> 
> ## Preserving History
> 
> I don't *think* the history of llvm-bazel is interesting enough to try 
> to merge it into the monorepo and I was planning to submit this as a 
> single patch, but please let me know if you disagree.
> 
> 
> ## Benefits to the community
> 
>   *
> 
>     Projects that depend on LLVM and use the Bazel build system can
>     avoid duplicating fragile effort. We'll spend more time
contributing
>     to LLVM instead :-D
> 
>   *
> 
>     Bazel is stricter than CMake in many ways (e.g. it requires that
>     even header dependencies be declared) and can catch layering issues
>     very easily. There's even an optional layering_check feature we
>     could turn on if its use would benefit the community. (though
>     currently the existing problematic layering makes it a burden to
>     maintain on our own). Even without that additional check, as I've
>     been keeping the Bazel build green, I've found and fixed a number
of
>     layering issues in the past couple weeks (e.g.
>     https://reviews.llvm.org/rGb49787df9a
>    
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>and
>     https://reviews.llvm.org/rGc17ae2916c
>    
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
> 
> 
> Here's a patch <https://reviews.llvm.org/D90352>adding the Bazel
build
> system. It's basically just `cp -r llvm-bazel/llvm-bazel 
> llvm-project/utils/bazel`.
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

On Fri, 30 Oct 2020 at 16:42, Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I think you've received some good feedback on this thread, but It also
> doesn't look like continuing discussion on this thread is going to lead
> to a consensus.
>
Hi Tom,

I started a thread on tier policy here:
http://lists.llvm.org/pipermail/llvm-dev/2020-October/146249.html

Maybe it will make the discussion here more focused after we reached a
consensus there.

--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201030/4d7ccb61/attachment.html>

I agree with Tom.  This seems like a natural time to use the decision making
process, and I encourage you to do so.  Some random points I’ve taken away from
the threads:

1) I don’t think this discussion has anything to do with the technical merits of
bazel over cmake.  There is empirically a community of people who would benefit
from this, and there is no proposal to replace cmake with bazel.  This is your
"We'll spend more time contributing to LLVM instead :-D” point.  I
would de-emphasize the technical points, because you’re not actually
evangelizing the technology here, you’re making a practical pitch.

2) I don’t think the comparison to GN is very important.  It is prior art, but
doesn’t mean that it is necessarily correlated to this decision.  However, it
doesn’t seem to me that GN has been a problem in practice for the community, so
perhaps there is something to learn from that.  For example, the
llvm/utils/GN/README.rst file clearly labels GN support as “experimental and
best effort”.  GN also seems maintained, and even has a "LLVM GN Syncbot”
that is doing stuff.

3) I think there is an important question (independent of Bazel) of “how do we
foster new things” and “how do we support things that are empirically important
to the community”?  We don’t want to burden all contributors to help out a few
of them, but keeping something “in tree” and marked experimental doesn’t seem
like a burden.

4) The big question is: Should this be in the mono-repo, a separate llvm
incubator project, or none of these?  What should the terms of support be, etc?


If you frame this carefully, I think we can make a decision quickly on this and
move on.  From my personal perspective, it seems pretty obvious that we should
take this, we just need to get the terms clear.  For example, if I check in a
cmake change and break a bazel builder, I shouldn’t be on the hook to fix it,
and arguably shouldn’t even get emailed about it.

-Chris


> On Oct 30, 2020, at 9:42 AM, Tom Stellard via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On 10/28/20 7:18 PM, Geoffrey Martin-Noble via llvm-dev wrote:
>> Hi all,
>> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR
in a side-directory in the monorepo, similar to the gn build.
> 
> Hi Geoffrey,
> 
> I think you've received some good feedback on this thread, but It also
doesn't look like continuing discussion on this thread is going to lead to a
consensus.  If you are still interested in making this change, I think it would
be best to escalate this proposal into a "proposal pitch" described
here[1].
> 
> There is also a template here[2] you can use.
> 
> [1]
https://github.com/llvm/llvm-www/blob/master/proposals/LP0001-LLVMDecisionMaking.md
> [2]
https://github.com/llvm/llvm-www/blob/master/proposals/LP0000-Template.md
> 
> -Tom
> 
>> Some of us have been working on open-source Bazel BUILD files for the
LLVM Project. You may have seen us hanging out in the #build-systems discord
channel. As you may know, Google uses Bazel internally and has maintained a
Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
we've got more and more OSS projects with a Bazel BUILD depending on LLVM
(e.g. IREE <https://github.com/google/iree>and TensorFlow
<https://github.com/tensorflow/tensorflow>). We're also not the only
ones using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've
borrowed from TF
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
Each of these projects has to jump through some weird hoops to keep their
version of the Bazel BUILD files in sync with the code, which requires some
fragile combination of scripts and human intervention. Instead, we'd like to
move general-purpose Bazel BUILD files into the LLVM Project monorepo. We expect
to follow the model of the GN build where these will be maintained by interested
contributors rather than expecting the general community to maintain them.
>> To facilitate and test this we've been developing a standalone
repository that just has the Bazel BUILD files. It symlinks together the
directory trees on top of a submodule as we would need in the monorepo to to
avoid in-tree BUILD files. The configuration is at
https://github.com/google/llvm-bazel. We now have those in a good place and
think they would be useful upstream.
>> # Details
>> ## What
>> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
<https://github.com/google/llvm-bazel/pull/72>) subprojects, potentially
expanding to others, as needed. Basically everything currently at
https://github.com/google/llvm-bazel.
>> ## Where
>> In https://github.com/google/llvm-bazelthe BUILD files live in a single
directory tree matching the structure of the overall llvm-project directory. For
users, @llvm-project is a single Bazel repository
<https://docs.bazel.build/versions/master/build-ref.html#repositories>that
includes both LLVM and MLIR subprojects. To maintain this structure, we would
probably want to put a `bazel` directory in the monorepo's utils directory
<https://github.com/llvm/llvm-project/tree/master/utils>, which currently
only contains a directory for arcanist. This is different from gn, which is
under the LLVM subproject's utils directory
<https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We could
similarly put the Bazel BUILD files under llvm/utils/bazel but have them be for
the entire llvm project (the subsets that are supported). This seems like an odd
structure to me, but I know that the CMake build for LLVM also builds the other
subprojects
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
so maybe this would be preferable.
>> Alternatively we could split each subproject into a separate Bazel
repository and put the Bazel build files under each subproject. I think this
fragments the configuration of the BUILD without much benefit.
>> ## Configurations
>> We currently have configurations for Linux GCC and Clang, MacOS GCC and
Clang, and Windows MSVC. Support for other configurations can be added
as-desired, but supporting all possible LLVM build configurations is not the
goal.
>> ## Support
>> Support would be similar to the gn build. Contributors could optionally
update the Bazel BUILD files as part of their patches, but would be under no
obligation to do so.
>> ## Preserving History
>> I don't *think* the history of llvm-bazel is interesting enough to
try to merge it into the monorepo and I was planning to submit this as a single
patch, but please let me know if you disagree.
>> ## Benefits to the community
>>  *
>>    Projects that depend on LLVM and use the Bazel build system can
>>    avoid duplicating fragile effort. We'll spend more time
contributing
>>    to LLVM instead :-D
>>  *
>>    Bazel is stricter than CMake in many ways (e.g. it requires that
>>    even header dependencies be declared) and can catch layering issues
>>    very easily. There's even an optional layering_check feature we
>>    could turn on if its use would benefit the community. (though
>>    currently the existing problematic layering makes it a burden to
>>    maintain on our own). Even without that additional check, as
I've
>>    been keeping the Bazel build green, I've found and fixed a
number of
>>    layering issues in the past couple weeks (e.g.
>>    https://reviews.llvm.org/rGb49787df9a
>>   
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>and
>>    https://reviews.llvm.org/rGc17ae2916c
>>   
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
>> Here's a patch <https://reviews.llvm.org/D90352>adding the
Bazel build system. It's basically just `cp -r llvm-bazel/llvm-bazel
llvm-project/utils/bazel`.
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi,

Doesn't an email like this maybe point to another problem that, if solved,
would make situations like this not a problem at all.
One would have 2 git repos.
The main one and an overlay one. Much like overlay filesystems work.
They would be linked in such a way that the developer would not need to
manually add dependencies but the git repos would be linked (linked at the
commit hash tree level) so that if one did a checkout from one, the correct
matching version of the other repo would also be checked out.
Then the main repo would just need a "readme", saying: To build with
Bazel
add this overlay repo.

But on the topic of having multiple build systems for LLVM.
If I was to wish to upstream a commit to LLVM, I would:
1) Want to know which build system that commit should work with.
2) Not have to make sure my commit worked with both build systems.

I worked on one project that had two build systems, and most of the time,
only one or the other build would actually work, because committers only
fixed the build system they actually used.

Kind Regards

James


On Wed, 28 Oct 2020 at 23:18, Geoffrey Martin-Noble via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi all,
>
> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
> side-directory in the monorepo, similar to the gn build.
>
> Some of us have been working on open-source Bazel BUILD files for the LLVM
> Project. You may have seen us hanging out in the #build-systems discord
> channel. As you may know, Google uses Bazel internally and has maintained a
> Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
> we've got more and more OSS projects with a Bazel BUILD depending on
LLVM
> (e.g. IREE <https://github.com/google/iree> and TensorFlow
> <https://github.com/tensorflow/tensorflow>). We're also not the
only ones
> using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've
borrowed
> from TF
>
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
> Each of these projects has to jump through some weird hoops to keep their
> version of the Bazel BUILD files in sync with the code, which requires some
> fragile combination of scripts and human intervention. Instead, we'd
like
> to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
> We expect to follow the model of the GN build where these will be
> maintained by interested contributors rather than expecting the general
> community to maintain them.
>
> To facilitate and test this we've been developing a standalone
repository
> that just has the Bazel BUILD files. It symlinks together the directory
> trees on top of a submodule as we would need in the monorepo to to avoid
> in-tree BUILD files. The configuration is at
> https://github.com/google/llvm-bazel. We now have those in a good place
> and think they would be useful upstream.
>
> # Details
>
> ## What
>
> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
> <https://github.com/google/llvm-bazel/pull/72>) subprojects,
potentially
> expanding to others, as needed. Basically everything currently at
> https://github.com/google/llvm-bazel.
>
> ## Where
>
> In https://github.com/google/llvm-bazel the BUILD files live in a single
> directory tree matching the structure of the overall llvm-project
> directory. For users, @llvm-project is a single Bazel repository
>
<https://docs.bazel.build/versions/master/build-ref.html#repositories>
> that includes both LLVM and MLIR subprojects. To maintain this structure,
> we would probably want to put a `bazel` directory in the monorepo's
utils
> directory <https://github.com/llvm/llvm-project/tree/master/utils>,
which
> currently only contains a directory for arcanist. This is different from
> gn, which is under the LLVM subproject's utils directory
> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
> could similarly put the Bazel BUILD files under llvm/utils/bazel but have
> them be for the entire llvm project (the subsets that are supported). This
> seems like an odd structure to me, but I know that the CMake build for LLVM
> also builds the other subprojects
>
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
> so maybe this would be preferable.
>
> Alternatively we could split each subproject into a separate Bazel
> repository and put the Bazel build files under each subproject. I think
> this fragments the configuration of the BUILD without much benefit.
>
> ## Configurations
>
> We currently have configurations for Linux GCC and Clang, MacOS GCC and
> Clang, and Windows MSVC. Support for other configurations can be added
> as-desired, but supporting all possible LLVM build configurations is not
> the goal.
>
> ## Support
>
> Support would be similar to the gn build. Contributors could optionally
> update the Bazel BUILD files as part of their patches, but would be under
> no obligation to do so.
>
> ## Preserving History
>
> I don't *think* the history of llvm-bazel is interesting enough to try
to
> merge it into the monorepo and I was planning to submit this as a single
> patch, but please let me know if you disagree.
>
> ## Benefits to the community
>
>    -
>
>    Projects that depend on LLVM and use the Bazel build system can avoid
>    duplicating fragile effort. We'll spend more time contributing to
LLVM
>    instead :-D
>    -
>
>    Bazel is stricter than CMake in many ways (e.g. it requires that even
>    header dependencies be declared) and can catch layering issues very
easily.
>    There's even an optional layering_check feature we could turn on if
its use
>    would benefit the community. (though currently the existing problematic
>    layering makes it a burden to maintain on our own). Even without that
>    additional check, as I've been keeping the Bazel build green,
I've found
>    and fixed a number of layering issues in the past couple weeks (e.g.
>    https://reviews.llvm.org/rGb49787df9a
>   
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
>    and https://reviews.llvm.org/rGc17ae2916c
>   
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
>
>
> Here's a patch <https://reviews.llvm.org/D90352> adding the Bazel
build
> system. It's basically just `cp -r llvm-bazel/llvm-bazel
> llvm-project/utils/bazel`.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201101/bfcad26c/attachment.html>

On Sun, Nov 1, 2020 at 2:00 AM James Courtier-Dutton <james.dutton at
gmail.com>
wrote:
> Hi,
>
> Doesn't an email like this maybe point to another problem that, if
solved,
> would make situations like this not a problem at all.
> One would have 2 git repos.
> The main one and an overlay one. Much like overlay filesystems work.
> They would be linked in such a way that the developer would not need to
> manually add dependencies but the git repos would be linked (linked at the
> commit hash tree level) so that if one did a checkout from one, the correct
> matching version of the other repo would also be checked out.
> Then the main repo would just need a "readme", saying: To build
with Bazel
> add this overlay repo.
>
> But on the topic of having multiple build systems for LLVM.
> If I was to wish to upstream a commit to LLVM, I would:
> 1) Want to know which build system that commit should work with.
> 2) Not have to make sure my commit worked with both build systems.
>
> I worked on one project that had two build systems, and most of the time,
> only one or the other build would actually work, because committers only
> fixed the build system they actually used.
>
> I believe this has been made pretty clear in this thread, but things are
somewhat scattered. In any accompanying documentation it would be made
clear that:
1. everything must build with CMake, as today.
2. commit authors have no responsibility to ensure something builds with
Bazel
> Kind Regards
>
> James
>
>
> On Wed, 28 Oct 2020 at 23:18, Geoffrey Martin-Noble via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi all,
>>
>> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR
in a
>> side-directory in the monorepo, similar to the gn build.
>>
>> Some of us have been working on open-source Bazel BUILD files for the
>> LLVM Project. You may have seen us hanging out in the #build-systems
>> discord channel. As you may know, Google uses Bazel internally and has
>> maintained a Bazel BUILD of LLVM for years. Especially with the
>> introduction of MLIR, we've got more and more OSS projects with a
Bazel
>> BUILD depending on LLVM (e.g. IREE
<https://github.com/google/iree> and
>> TensorFlow <https://github.com/tensorflow/tensorflow>). We're
also not
>> the only ones using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM
that
>> they've borrowed from TF
>>
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
>> Each of these projects has to jump through some weird hoops to keep
their
>> version of the Bazel BUILD files in sync with the code, which requires
some
>> fragile combination of scripts and human intervention. Instead,
we'd
>> like to move general-purpose Bazel BUILD files into the LLVM Project
>> monorepo. We expect to follow the model of the GN build where these
will be
>> maintained by interested contributors rather than expecting the general
>> community to maintain them.
>>
>> To facilitate and test this we've been developing a standalone
repository
>> that just has the Bazel BUILD files. It symlinks together the directory
>> trees on top of a submodule as we would need in the monorepo to to
avoid
>> in-tree BUILD files. The configuration is at
>> https://github.com/google/llvm-bazel. We now have those in a good place
>> and think they would be useful upstream.
>>
>> # Details
>>
>> ## What
>>
>> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
>> <https://github.com/google/llvm-bazel/pull/72>) subprojects,
potentially
>> expanding to others, as needed. Basically everything currently at
>> https://github.com/google/llvm-bazel.
>>
>> ## Where
>>
>> In https://github.com/google/llvm-bazel the BUILD files live in a
single
>> directory tree matching the structure of the overall llvm-project
>> directory. For users, @llvm-project is a single Bazel repository
>>
<https://docs.bazel.build/versions/master/build-ref.html#repositories>
>> that includes both LLVM and MLIR subprojects. To maintain this
structure,
>> we would probably want to put a `bazel` directory in the monorepo's
utils
>> directory
<https://github.com/llvm/llvm-project/tree/master/utils>,
>> which currently only contains a directory for arcanist. This is
different
>> from gn, which is under the LLVM subproject's utils directory
>> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>.
We
>> could similarly put the Bazel BUILD files under llvm/utils/bazel but
have
>> them be for the entire llvm project (the subsets that are supported).
This
>> seems like an odd structure to me, but I know that the CMake build for
LLVM
>> also builds the other subprojects
>>
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
>> so maybe this would be preferable.
>>
>> Alternatively we could split each subproject into a separate Bazel
>> repository and put the Bazel build files under each subproject. I think
>> this fragments the configuration of the BUILD without much benefit.
>>
>> ## Configurations
>>
>> We currently have configurations for Linux GCC and Clang, MacOS GCC and
>> Clang, and Windows MSVC. Support for other configurations can be added
>> as-desired, but supporting all possible LLVM build configurations is
not
>> the goal.
>>
>> ## Support
>>
>> Support would be similar to the gn build. Contributors could optionally
>> update the Bazel BUILD files as part of their patches, but would be
under
>> no obligation to do so.
>>
>> ## Preserving History
>>
>> I don't *think* the history of llvm-bazel is interesting enough to
try to
>> merge it into the monorepo and I was planning to submit this as a
single
>> patch, but please let me know if you disagree.
>>
>> ## Benefits to the community
>>
>>    -
>>
>>    Projects that depend on LLVM and use the Bazel build system can
avoid
>>    duplicating fragile effort. We'll spend more time contributing
to LLVM
>>    instead :-D
>>    -
>>
>>    Bazel is stricter than CMake in many ways (e.g. it requires that
even
>>    header dependencies be declared) and can catch layering issues very
easily.
>>    There's even an optional layering_check feature we could turn on
if its use
>>    would benefit the community. (though currently the existing
problematic
>>    layering makes it a burden to maintain on our own). Even without
that
>>    additional check, as I've been keeping the Bazel build green,
I've found
>>    and fixed a number of layering issues in the past couple weeks (e.g.
>>    https://reviews.llvm.org/rGb49787df9a
>>   
<https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
>>    and https://reviews.llvm.org/rGc17ae2916c
>>   
<https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>
>>    ).
>>
>>
>> Here's a patch <https://reviews.llvm.org/D90352> adding the
Bazel build
>> system. It's basically just `cp -r llvm-bazel/llvm-bazel
>> llvm-project/utils/bazel`.
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201101/589fe26b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3992 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201101/589fe26b/attachment-0001.bin>