Chris Tetreault via llvm-dev
2020-Jun-18 19:51 UTC
[llvm-dev] RFC: A top level monorepo CMake file
I am in favor of this change because having the "root" CMakeLists.txt
be in the llvm subfolder breaks some IDEs. Specifically, if you try to use
Visual Studio's CMake integration, you will only see the llvm subproject in
the file browser pane. Visual Studio works fine if you manually generate and
MSBuild project, but I keep hearing about how much faster everything is with
Ninja, and this is a major issue that prevents me from trying to use it.
Thanks,
Christopher Tetreault
From: llvm-dev <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces
at lists.llvm.org>> On Behalf Of Louis Dionne via llvm-dev
Sent: Thursday, June 18, 2020 10:58 AM
To: LLVM Developers' List <llvm-dev at lists.llvm.org<mailto:llvm-dev
at lists.llvm.org>>
Cc: Libc++ Dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at
lists.llvm.org>>
Subject: [EXT] [llvm-dev] RFC: A top level monorepo CMake file
Hi folks,
Building any LLVM project currently requires invoking CMake inside
<monorepo-root>/llvm, while setting the projects to enable in the
LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing for
the LLVM subproject happens even when one doesn't really need or want it.
It's also not great from a build hygiene perspective, as LLVM globally sets
some flags and subprojects pick them up, when they don't really mean to. For
example, see this workaround:
https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
where we need to account for some flags that might have been set globally by
LLVM.
I'm not sure about other projects, however this is quite problematic for
projects part of the C++ runtime (libc++/libc++abi/libunwind). Indeed, we often
try to build those projects targetting not widely supported platforms, where the
overall LLVM build doesn't work. For example, trying to use the
LLVM_ENABLE_PROJECTS approach for building libc++ for Apple's DriverKit
environment doesn't work, since it has a few unusual things that the LLVM
build chokes on. However, building libc++ standalone works just fine because it
has far fewer requirements. It's also not just an issue of working vs not
working: because of global flag pollution, building libc++ standalone and as
part of the rest of LLVM can result in slightly different flags being used,
which could cause important and hard-to-diagnose issues.
Hence, I think we should introduce a way to build LLVM projects (or at least the
runtimes) without going through <monorepo-root>/llvm/CMakeLists.txt. What
I suggest is to have a top-level <monorepo-root>/CMakeLists.txt whose sole
job is to include subprojects. We could also place basic LLVM-wide things like
the check for the minimum CMake version there. More specifically, I would like
to be able to do:
$ cd <monorepo-root>
$ mkdir build
$ (cd build && cmake <monorepo-root>
-DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
Pretty much the only difference with today is that you'd use `cmake
<monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
Like I said, this is a problem for the runtime projects, but I'm not sure
about other projects. For the runtime projects, another option would be to only
allow standalone builds. However, the runtime projects are often built in
lockstep, and so running three CMake commands when one would suffice is both
annoying and also an easy way to screw things up. Furthermore, the current
standalone builds add complexity to the projects, because they require the
ability to point to arbitrary headers/libraries from the other projects, when we
really always want to point to the just-built ones.
Relationship with Petr Hosek's "Runtimes" build
---------------------------------------------------------------
What I'm proposing isn't a replacement for itl. The "Runtimes"
build can be seen as a driver that sets up the individual
libc++/libc++abi/libunwind builds with the just-built toolchain, and for the
provided targets. That's really great, however it is built *on top of* the
basic libc++/libc++abi/libunwind builds. So basically, after my proposal, the
"Runtimes" build could simply build all elements from the runtime with
a single CMake invocation, as opposed to multiple invocations.
Thoughts?
Louis
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200618/62a0a330/attachment.html>
Aaron Ballman via llvm-dev
2020-Jun-18 20:43 UTC
[llvm-dev] RFC: A top level monorepo CMake file
On Thu, Jun 18, 2020 at 3:51 PM Chris Tetreault via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > I am in favor of this change because having the “root” CMakeLists.txt be in the llvm subfolder breaks some IDEs. Specifically, if you try to use Visual Studio’s CMake integration, you will only see the llvm subproject in the file browser pane. Visual Studio works fine if you manually generate and MSBuild project, but I keep hearing about how much faster everything is with Ninja, and this is a major issue that prevents me from trying to use it.Strong +1 to this use case. The monorepo has made working on Clang in Visual Studio considerably harder and I believe this change will fix a lot of the remaining issues I have. ~Aaron> > > > Thanks, > > Christopher Tetreault > > > > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Louis Dionne via llvm-dev > Sent: Thursday, June 18, 2020 10:58 AM > To: LLVM Developers' List <llvm-dev at lists.llvm.org> > Cc: Libc++ Dev <libcxx-dev at lists.llvm.org> > Subject: [EXT] [llvm-dev] RFC: A top level monorepo CMake file > > > > Hi folks, > > > > Building any LLVM project currently requires invoking CMake inside <monorepo-root>/llvm, while setting the projects to enable in the LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing for the LLVM subproject happens even when one doesn't really need or want it. It's also not great from a build hygiene perspective, as LLVM globally sets some flags and subprojects pick them up, when they don't really mean to. For example, see this workaround: https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507, where we need to account for some flags that might have been set globally by LLVM. > > > > I'm not sure about other projects, however this is quite problematic for projects part of the C++ runtime (libc++/libc++abi/libunwind). Indeed, we often try to build those projects targetting not widely supported platforms, where the overall LLVM build doesn't work. For example, trying to use the LLVM_ENABLE_PROJECTS approach for building libc++ for Apple's DriverKit environment doesn't work, since it has a few unusual things that the LLVM build chokes on. However, building libc++ standalone works just fine because it has far fewer requirements. It's also not just an issue of working vs not working: because of global flag pollution, building libc++ standalone and as part of the rest of LLVM can result in slightly different flags being used, which could cause important and hard-to-diagnose issues. > > > > Hence, I think we should introduce a way to build LLVM projects (or at least the runtimes) without going through <monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a top-level <monorepo-root>/CMakeLists.txt whose sole job is to include subprojects. We could also place basic LLVM-wide things like the check for the minimum CMake version there. More specifically, I would like to be able to do: > > > > $ cd <monorepo-root> > > $ mkdir build > > $ (cd build && cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS="<projects-to-enable>") > > > > Pretty much the only difference with today is that you'd use `cmake <monorepo-root>` instead of `cmake <monorepo-root>/llvm`. > > > > Like I said, this is a problem for the runtime projects, but I'm not sure about other projects. For the runtime projects, another option would be to only allow standalone builds. However, the runtime projects are often built in lockstep, and so running three CMake commands when one would suffice is both annoying and also an easy way to screw things up. Furthermore, the current standalone builds add complexity to the projects, because they require the ability to point to arbitrary headers/libraries from the other projects, when we really always want to point to the just-built ones. > > > > Relationship with Petr Hosek's "Runtimes" build > > --------------------------------------------------------------- > > What I'm proposing isn't a replacement for itl. The "Runtimes" build can be seen as a driver that sets up the individual libc++/libc++abi/libunwind builds with the just-built toolchain, and for the provided targets. That's really great, however it is built *on top of* the basic libc++/libc++abi/libunwind builds. So basically, after my proposal, the "Runtimes" build could simply build all elements from the runtime with a single CMake invocation, as opposed to multiple invocations. > > > > Thoughts? > > Louis > > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev