llvm dev - Oct 2020 - Contributing Bazel BUILD files similar to gn

I think Renato has articulated quite well some concerns I have about this but
was unable to express. I would very much prefer if we just focus on using CMake
effectively.

Thanks,
   Christopher Tetreault

From: llvm-dev <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces
at lists.llvm.org>> On Behalf Of Renato Golin via llvm-dev
Sent: Thursday, October 29, 2020 9:06 AM
To: tstellar at redhat.com<mailto:tstellar at redhat.com>
Cc: Mehdi Amini <aminim at google.com<mailto:aminim at google.com>>;
LLVM Dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at
lists.llvm.org>>; Stella Laurenzo <laurenzo at
google.com<mailto:laurenzo at google.com>>; Tres Popp <tpopp at
google.com<mailto:tpopp at google.com>>; Geoffrey Martin-Noble <gcmn
at google.com<mailto:gcmn at google.com>>; Thomas Joerg <tjoerg at
google.com<mailto:tjoerg at google.com>>
Subject: [EXT] Re: [llvm-dev] Contributing Bazel BUILD files similar to gn

On Thu, 29 Oct 2020 at 15:23, Tom Stellard via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
I'm a little concerned about having two 'unsupported' buildsystems
living in tree, and I'm not sure what would stop us from continuing to
add more.  I would feel better if we had a set of guidelines to define
the criteria for adding a new buildsytem and also criteria for when we
can remove them.

I have used Bazel and it doesn't seem to map well to CMake. It seems to be
in between CMake and Ninja with a lot of hard-coded dependencies that are
cumbersome to keep updating. I'm by no means an expert, and I could very
well be wrong, but supporting more than one build system is not trivial
(remember the autoconf days?).

For example, when trying to implement the same logic on both will not be
trivial. So, whenever we want to add some functionality or improve how we build
LLVM with one system, we'll have to do so in multiple build systems that do
not easily match each other. If we don't try to match functionality,
we'll segregate the community, because people will be able to do X on build
system A but not B, and the similar features cluster together and then we have
essentially two projects built from the same source code.

Testing this, or worse, trying to fix a buildbot that is built with Bazel (and
having to install Java JDK and all its dependencies) on potentially a hardware
that you do not have access to, will be a nightmare to debug. The nature of
post-commit testing, revert and review of LLVM will not make that simpler.
Unless we treat the Bazel build as "not our problem" (which defeats
the point of having it?).

To make matters worse, our CMake files are not simple, and do not do all of the
things we want them to do in the way we understand completely. There is a lot of
kludge that we carry and with that comes in two categories: the things that we
hate and would love to fix, and the things that are fixes that we have no idea
are there. The former are the reasons why people want to start a new build
system, the latter is why they soon realise that was a mistake (insert XKCD joke
here).

If the Bazel files can be completely ignored, then it's just more clutter.
But if other projects start to use more different build systems and we start
packing them all in LLVM, then we'll have a hard time knowing what we build
how. I can't really see this scaling.

Two-cents worth.
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/2e34dc38/attachment-0001.html>

On Thu, Oct 29, 2020 at 12:29 PM Chris Tetreault via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I think Renato has articulated quite well some concerns I have about this
> but was unable to express. I would very much prefer if we just focus on
> using CMake effectively.
>
...
> For example, when trying to implement the same logic on both will not be
> trivial. So, whenever we want to add some functionality or improve how we
> build LLVM with one system, we'll have to do so in multiple build
systems
> that do not easily match each other.
>
Google already does all of this work, and has for years. I think it is fair
to say that it hasn't been a burden on the community.

> If we don't try to match functionality, we'll segregate the
community,
> because people will be able to do X on build system A but not B, and the
> similar features cluster together and then we have essentially two projects
> built from the same source code.
>
>
As long as we keep CMake as the canonical system everything will be fine.
It works perfectly well today, except that not everyone gets to see or use
the bazel files. They exist right now; they work right now; and it hasn't
been a burden on anyone but the people who care about bazel.

> Testing this, or worse, trying to fix a buildbot that is built with Bazel
> (and having to install Java JDK and all its dependencies) on potentially a
> hardware that you do not have access to, will be a nightmare to debug. The
> nature of post-commit testing, revert and review of LLVM will not make that
> simpler. Unless we treat the Bazel build as "not our problem"
(which
> defeats the point of having it?).
>
>
Google makes it work like this today, with the rest of the project treating
it as "not our problem" because they don't even see that they
exist. The
build bot issues would be real, but I think surmountable, given that Google
already cleans up the bazel files, it just doesn't push them. Perhaps an
explicit policy that cmake folks don't have to update the bazel files would
be helpful.

> To make matters worse, our CMake files are not simple, and do not do all
> of the things we want them to do in the way we understand completely. There
> is a lot of kludge that we carry and with that comes in two categories: the
> things that we hate and would love to fix, and the things that are fixes
> that we have no idea are there. The former are the reasons why people want
> to start a new build system, the latter is why they soon realise that was a
> mistake (insert XKCD joke here).
>
It wouldn't be starting a new build system, it would be making a
pre-existing, already extremely well functioning one, available to more
people.

I can definitely see folks who use cmake not wanting more hassle--that may
be a valid reason not to do it. But "it won't work" or
"it's hard to keep
up" or "it's too complicated" seem well refuted by a
multi-year existence
proof.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/fa2128d4/attachment-0001.html>

Not seeing Bazel at https://packages.ubuntu.com/.

Glad to see that the current required cmake version is being met with 
Ubuntu 20.04 and later.

On Thu, Oct 29, 2020 at 1:14 PM Sterling Augustine via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Thu, Oct 29, 2020 at 12:29 PM Chris Tetreault via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> I think Renato has articulated quite well some concerns I have about
this
>> but was unable to express. I would very much prefer if we just focus on
>> using CMake effectively.
>>
> ...
>
>> For example, when trying to implement the same logic on both will not
be
>> trivial. So, whenever we want to add some functionality or improve how
we
>> build LLVM with one system, we'll have to do so in multiple build
systems
>> that do not easily match each other.
>>
>
> Google already does all of this work, and has for years. I think it is
> fair to say that it hasn't been a burden on the community.
>
As far as I can tell, the only burden would be the existence of bazel files
in the repo, but scoped within a directory most will never look at and can
ignore. The issue of extra files seems of most concern to anyone who is
tracking/optimizing repo size or file and commit count.

>
> I can definitely see folks who use cmake not wanting more hassle--that may
> be a valid reason not to do it.
>
What would be the hassle to cmake users?

Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/8c7dfe66/attachment.html>

On Thu, 29 Oct 2020 at 20:14, Sterling Augustine <saugustine at
google.com>
wrote:
> To make matters worse, our CMake files are not simple, and do not do all
>> of the things we want them to do in the way we understand completely.
There
>> is a lot of kludge that we carry and with that comes in two categories:
the
>> things that we hate and would love to fix, and the things that are
fixes
>> that we have no idea are there. The former are the reasons why people
want
>> to start a new build system, the latter is why they soon realise that
was a
>> mistake (insert XKCD joke here).
>>
>
> It wouldn't be starting a new build system, it would be making a
> pre-existing, already extremely well functioning one, available to more
> people.
>
> I can definitely see folks who use cmake not wanting more hassle--that may
> be a valid reason not to do it. But "it won't work" or
"it's hard to keep
> up" or "it's too complicated" seem well refuted by a
multi-year existence
> proof.
>
That is definitely not what I meant here.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/86264200/attachment.html>