Chris Tetreault via llvm-dev
2020-Oct-29 20:24 UTC
[llvm-dev] Contributing Bazel BUILD files similar to gn
The problem is that once it’s in community LLVM, it becomes the community’s problem. The expectation is that individual contributors do not break anything in upstream. Why else would you contribute it to the LLVM monorepo? If the goal is just to enable external-to-google orgs to collaborate on it, why not contribute it as a new repo separate from LLVM? You wouldn’t need to ask anybody’s permission to do this. From: Sterling Augustine <saugustine at google.com> Sent: Thursday, October 29, 2020 1:14 PM To: Chris Tetreault <ctetreau at quicinc.com> Cc: Renato Golin <rengolin at gmail.com>; tstellar at redhat.com; Mehdi Amini <aminim at google.com>; LLVM Dev <llvm-dev at lists.llvm.org>; Stella Laurenzo <laurenzo at google.com>; Tres Popp <tpopp at google.com>; Geoffrey Martin-Noble <gcmn at google.com>; Thomas Joerg <tjoerg at google.com> Subject: [EXT] Re: [llvm-dev] Contributing Bazel BUILD files similar to gn On Thu, Oct 29, 2020 at 12:29 PM Chris Tetreault via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: I think Renato has articulated quite well some concerns I have about this but was unable to express. I would very much prefer if we just focus on using CMake effectively. ... For example, when trying to implement the same logic on both will not be trivial. So, whenever we want to add some functionality or improve how we build LLVM with one system, we'll have to do so in multiple build systems that do not easily match each other. Google already does all of this work, and has for years. I think it is fair to say that it hasn't been a burden on the community. If we don't try to match functionality, we'll segregate the community, because people will be able to do X on build system A but not B, and the similar features cluster together and then we have essentially two projects built from the same source code. As long as we keep CMake as the canonical system everything will be fine. It works perfectly well today, except that not everyone gets to see or use the bazel files. They exist right now; they work right now; and it hasn't been a burden on anyone but the people who care about bazel. Testing this, or worse, trying to fix a buildbot that is built with Bazel (and having to install Java JDK and all its dependencies) on potentially a hardware that you do not have access to, will be a nightmare to debug. The nature of post-commit testing, revert and review of LLVM will not make that simpler. Unless we treat the Bazel build as "not our problem" (which defeats the point of having it?). Google makes it work like this today, with the rest of the project treating it as "not our problem" because they don't even see that they exist. The build bot issues would be real, but I think surmountable, given that Google already cleans up the bazel files, it just doesn't push them. Perhaps an explicit policy that cmake folks don't have to update the bazel files would be helpful. To make matters worse, our CMake files are not simple, and do not do all of the things we want them to do in the way we understand completely. There is a lot of kludge that we carry and with that comes in two categories: the things that we hate and would love to fix, and the things that are fixes that we have no idea are there. The former are the reasons why people want to start a new build system, the latter is why they soon realise that was a mistake (insert XKCD joke here). It wouldn't be starting a new build system, it would be making a pre-existing, already extremely well functioning one, available to more people. I can definitely see folks who use cmake not wanting more hassle--that may be a valid reason not to do it. But "it won't work" or "it's hard to keep up" or "it's too complicated" seem well refuted by a multi-year existence proof. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/f5ec99af/attachment.html>
Mehdi AMINI via llvm-dev
2020-Oct-29 20:59 UTC
[llvm-dev] Contributing Bazel BUILD files similar to gn
On Thu, Oct 29, 2020 at 1:24 PM Chris Tetreault via llvm-dev < llvm-dev at lists.llvm.org> wrote:> The problem is that once it’s in community LLVM, it becomes the > community’s problem. The expectation is that individual contributors do > not break anything in upstream. >I would expect that the community by now has concrete experience with `gn` gained over a few years demonstrating that this hasn't been a problem to have this in-tree, without a burden of support on the community. In particular, I think that a salient point is the guarantee that no public bot would be testing it (I mean here by "no public bot" that no bot would email you when you break it).> Why else would you contribute it to the LLVM monorepo? If the goal is just > to enable external-to-google orgs to collaborate on it, why not contribute > it as a new repo separate from LLVM? You wouldn’t need to ask anybody’s > permission to do this. >Yes, we could do this, and you are correct that in many cases a motivation to upstream a component is to make sure it is maintained by the community and works out of the box. In this case it is slightly different: we are OK with people to break this. We are already maintaining these files out-of-tree for our own purposes, and this has been the case for years as Sterling mentions. I would even suspect that for Google internal build integration, it is actually easier to have these files internal only rather than unsupported upstream. So why are we doing it? I mentioned this in another answer: this is mainly to provide a collaboration space for the support of OSS projects using Bazel interested to use LLVM (and some subprojects). Having them in-tree means that we can publish every day (or more) a git hash that we validate with Bazel on private bots (like `gn`) and every project can use to clone the LLVM monorepo and integrate in their build flow easily. Another repo, submodules, etc. are not making this possible / practical.> > > *From:* Sterling Augustine <saugustine at google.com> > *Sent:* Thursday, October 29, 2020 1:14 PM > *To:* Chris Tetreault <ctetreau at quicinc.com> > *Cc:* Renato Golin <rengolin at gmail.com>; tstellar at redhat.com; Mehdi Amini > <aminim at google.com>; LLVM Dev <llvm-dev at lists.llvm.org>; Stella Laurenzo < > laurenzo at google.com>; Tres Popp <tpopp at google.com>; Geoffrey Martin-Noble > <gcmn at google.com>; Thomas Joerg <tjoerg at google.com> > *Subject:* [EXT] Re: [llvm-dev] Contributing Bazel BUILD files similar to > gn > > > > On Thu, Oct 29, 2020 at 12:29 PM Chris Tetreault via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > I think Renato has articulated quite well some concerns I have about this > but was unable to express. I would very much prefer if we just focus on > using CMake effectively. > > ... > > For example, when trying to implement the same logic on both will not be > trivial. So, whenever we want to add some functionality or improve how we > build LLVM with one system, we'll have to do so in multiple build systems > that do not easily match each other. > > > > Google already does all of this work, and has for years. I think it is > fair to say that it hasn't been a burden on the community. > > > > If we don't try to match functionality, we'll segregate the community, > because people will be able to do X on build system A but not B, and the > similar features cluster together and then we have essentially two projects > built from the same source code. > > > > As long as we keep CMake as the canonical system everything will be fine. > It works perfectly well today, except that not everyone gets to see or use > the bazel files. They exist right now; they work right now; and it hasn't > been a burden on anyone but the people who care about bazel. > > > > Testing this, or worse, trying to fix a buildbot that is built with Bazel > (and having to install Java JDK and all its dependencies) on potentially a > hardware that you do not have access to, will be a nightmare to debug. The > nature of post-commit testing, revert and review of LLVM will not make that > simpler. Unless we treat the Bazel build as "not our problem" (which > defeats the point of having it?). > > > > Google makes it work like this today, with the rest of the project > treating it as "not our problem" because they don't even see that they > exist. The build bot issues would be real, but I think surmountable, given > that Google already cleans up the bazel files, it just doesn't push them. > Perhaps an explicit policy that cmake folks don't have to update the bazel > files would be helpful. > > > > To make matters worse, our CMake files are not simple, and do not do all > of the things we want them to do in the way we understand completely. There > is a lot of kludge that we carry and with that comes in two categories: the > things that we hate and would love to fix, and the things that are fixes > that we have no idea are there. The former are the reasons why people want > to start a new build system, the latter is why they soon realise that was a > mistake (insert XKCD joke here). > > > > It wouldn't be starting a new build system, it would be making a > pre-existing, already extremely well functioning one, available to more > people. > > > > I can definitely see folks who use cmake not wanting more hassle--that may > be a valid reason not to do it. But "it won't work" or "it's hard to keep > up" or "it's too complicated" seem well refuted by a multi-year existence > proof. > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/76e41919/attachment.html>