llvm dev - Jun 2020 - RFC: A top level monorepo CMake file

On 06/18/2020 11:27 AM, Steven Wu via llvm-dev wrote:
> I like the proposal but I would like to go even further. If we are going to
create a top level CMake file, we should just go ahead and eliminate all the
standalone build configuration. The standalone build should just be `cmake
<monorepo-root> -DLLVM_ENABLE_PROJECTS=standalone-project ...`. That means
less build configuration to maintain which is always good.
> 
Eliminating stand-alone builds would be an inconvenience for
us in Fedora, since this is how we build LLVM packages.
However, I think we may have different definitions of what a
stand-alone build is.

Does your proposal mean that
`cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=lld ..` would build lld
only without llvm and be a replacement for what I consider to
be a stand-alone build, which is `cmake <monorepo-root>/lld ...`

-Tom
> That also means that we can refactor some of the cmake modules to put into
the top level directories so sub-projects can just reuse them without duping the
code worrying they are not available in the standalone build today.
> 
> Steven
> 
>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>> Hi folks,
>>
>> Building any LLVM project currently requires invoking CMake inside
<monorepo-root>/llvm, while setting the projects to enable in the
LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing for
the LLVM subproject happens even when one doesn't really need or want it.
It's also not great from a build hygiene perspective, as LLVM globally sets
some flags and subprojects pick them up, when they don't really mean to. For
example, see this workaround:
https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
where we need to account for some flags that might have been set globally by
LLVM.
>>
>> I'm not sure about other projects, however this is quite
problematic for projects part of the C++ runtime (libc++/libc++abi/libunwind).
Indeed, we often try to build those projects targetting not widely supported
platforms, where the overall LLVM build doesn't work. For example, trying to
use the LLVM_ENABLE_PROJECTS approach for building libc++ for Apple's
DriverKit environment doesn't work, since it has a few unusual things that
the LLVM build chokes on. However, building libc++ standalone works just fine
because it has far fewer requirements. It's also not just an issue of
working vs not working: because of global flag pollution, building libc++
standalone and as part of the rest of LLVM can result in slightly different
flags being used, which could cause important and hard-to-diagnose issues.
>>
>> Hence, I think we should introduce a way to build LLVM projects (or at
least the runtimes) without going through
<monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a top-level
<monorepo-root>/CMakeLists.txt whose sole job is to include subprojects.
We could also place basic LLVM-wide things like the check for the minimum CMake
version there. More specifically, I would like to be able to do:
>>
>>     $ cd <monorepo-root>
>>     $ mkdir build
>>     $ (cd build && cmake <monorepo-root>
-DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
>>
>> Pretty much the only difference with today is that you'd use `cmake
<monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
>>
>> Like I said, this is a problem for the runtime projects, but I'm
not sure about other projects. For the runtime projects, another option would be
to only allow standalone builds. However, the runtime projects are often built
in lockstep, and so running three CMake commands when one would suffice is both
annoying and also an easy way to screw things up. Furthermore, the current
standalone builds add complexity to the projects, because they require the
ability to point to arbitrary headers/libraries from the other projects, when we
really always want to point to the just-built ones.
>>
>> Relationship with Petr Hosek's "Runtimes" build
>> ---------------------------------------------------------------
>> What I'm proposing isn't a replacement for itl. The
"Runtimes" build can be seen as a driver that sets up the individual
libc++/libc++abi/libunwind builds with the just-built toolchain, and for the
provided targets. That's really great, however it is built *on top of* the
basic libc++/libc++abi/libunwind builds. So basically, after my proposal, the
"Runtimes" build could simply build all elements from the runtime with
a single CMake invocation, as opposed to multiple invocations.
>>
>> Thoughts?
>> Louis
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

> On Jun 18, 2020, at 2:25 PM, Tom Stellard <tstellar at redhat.com>
wrote:
> 
> On 06/18/2020 11:27 AM, Steven Wu via llvm-dev wrote:
>> I like the proposal but I would like to go even further. If we are
going to create a top level CMake file, we should just go ahead and eliminate
all the standalone build configuration. The standalone build should just be
`cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=standalone-project ...`.
That means less build configuration to maintain which is always good.
>> 
> 
> Eliminating stand-alone builds would be an inconvenience for
> us in Fedora, since this is how we build LLVM packages.
> However, I think we may have different definitions of what a
> stand-alone build is.
> 
> Does your proposal mean that
> `cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=lld ..` would build lld
> only without llvm and be a replacement for what I consider to
> be a stand-alone build, which is `cmake <monorepo-root>/lld ...`
Let me clear up the terminology here. In current llvm-project, we have a full
build and standalone build. Let say for libcxx (using libcxx or other runtime
libraries because they can build without anything from llvm subdirectory).

This is a full build that can produce libcxx dylib:
`cmake <monorepo-root>/llvm -DLLVM_ENABLE_PROJECTS=llvm;libcxx
-DLLVM_DISTRIBUTION_COMPONENTS=libcxx.dylib ...`

This is a standalone build that doesn't pull in anything from llvm
directory:
`cmake <monorepo-root>/libcxx -D...`

Those two cmake configurations are going down different path and can result in
different behavior if not careful. The goal is just turning those into one
configuration. You can still do a standalone build of runtime library by
enabling just that project and it is going to produce the same build
configuration as the full build.

Steven
> 
> -Tom
> 
>> That also means that we can refactor some of the cmake modules to put
into the top level directories so sub-projects can just reuse them without
duping the code worrying they are not available in the standalone build today.
>> 
>> Steven
>> 
>>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
>>> 
>>> Hi folks,
>>> 
>>> Building any LLVM project currently requires invoking CMake inside
<monorepo-root>/llvm, while setting the projects to enable in the
LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing for
the LLVM subproject happens even when one doesn't really need or want it.
It's also not great from a build hygiene perspective, as LLVM globally sets
some flags and subprojects pick them up, when they don't really mean to. For
example, see this workaround:
https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
where we need to account for some flags that might have been set globally by
LLVM.
>>> 
>>> I'm not sure about other projects, however this is quite
problematic for projects part of the C++ runtime (libc++/libc++abi/libunwind).
Indeed, we often try to build those projects targetting not widely supported
platforms, where the overall LLVM build doesn't work. For example, trying to
use the LLVM_ENABLE_PROJECTS approach for building libc++ for Apple's
DriverKit environment doesn't work, since it has a few unusual things that
the LLVM build chokes on. However, building libc++ standalone works just fine
because it has far fewer requirements. It's also not just an issue of
working vs not working: because of global flag pollution, building libc++
standalone and as part of the rest of LLVM can result in slightly different
flags being used, which could cause important and hard-to-diagnose issues.
>>> 
>>> Hence, I think we should introduce a way to build LLVM projects (or
at least the runtimes) without going through
<monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a top-level
<monorepo-root>/CMakeLists.txt whose sole job is to include subprojects.
We could also place basic LLVM-wide things like the check for the minimum CMake
version there. More specifically, I would like to be able to do:
>>> 
>>>    $ cd <monorepo-root>
>>>    $ mkdir build
>>>    $ (cd build && cmake <monorepo-root>
-DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
>>> 
>>> Pretty much the only difference with today is that you'd use
`cmake <monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
>>> 
>>> Like I said, this is a problem for the runtime projects, but
I'm not sure about other projects. For the runtime projects, another option
would be to only allow standalone builds. However, the runtime projects are
often built in lockstep, and so running three CMake commands when one would
suffice is both annoying and also an easy way to screw things up. Furthermore,
the current standalone builds add complexity to the projects, because they
require the ability to point to arbitrary headers/libraries from the other
projects, when we really always want to point to the just-built ones.
>>> 
>>> Relationship with Petr Hosek's "Runtimes" build
>>> ---------------------------------------------------------------
>>> What I'm proposing isn't a replacement for itl. The
"Runtimes" build can be seen as a driver that sets up the individual
libc++/libc++abi/libunwind builds with the just-built toolchain, and for the
provided targets. That's really great, however it is built *on top of* the
basic libc++/libc++abi/libunwind builds. So basically, after my proposal, the
"Runtimes" build could simply build all elements from the runtime with
a single CMake invocation, as opposed to multiple invocations.
>>> 
>>> Thoughts?
>>> Louis
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>

On Thu, Jun 18, 2020 at 2:41 PM Steven Wu via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>
> > On Jun 18, 2020, at 2:25 PM, Tom Stellard <tstellar at
redhat.com> wrote:
> >
> > On 06/18/2020 11:27 AM, Steven Wu via llvm-dev wrote:
> >> I like the proposal but I would like to go even further. If we are
> going to create a top level CMake file, we should just go ahead and
> eliminate all the standalone build configuration. The standalone build
> should just be `cmake <monorepo-root>
> -DLLVM_ENABLE_PROJECTS=standalone-project ...`. That means less build
> configuration to maintain which is always good.
> >>
> >
> > Eliminating stand-alone builds would be an inconvenience for
> > us in Fedora, since this is how we build LLVM packages.
> > However, I think we may have different definitions of what a
> > stand-alone build is.
> >
> > Does your proposal mean that
> > `cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=lld ..` would
build lld
> > only without llvm and be a replacement for what I consider to
> > be a stand-alone build, which is `cmake <monorepo-root>/lld ...`
> Let me clear up the terminology here. In current llvm-project, we have a
> full build and standalone build. Let say for libcxx (using libcxx or other
> runtime libraries because they can build without anything from llvm
> subdirectory).
>
> This is a full build that can produce libcxx dylib:
> `cmake <monorepo-root>/llvm -DLLVM_ENABLE_PROJECTS=llvm;libcxx
> -DLLVM_DISTRIBUTION_COMPONENTS=libcxx.dylib ...`
>
> This is a standalone build that doesn't pull in anything from llvm
> directory:
> `cmake <monorepo-root>/libcxx -D...`
>
> Those two cmake configurations are going down different path and can
> result in different behavior if not careful. The goal is just turning those
> into one configuration. You can still do a standalone build of runtime
> library by enabling just that project and it is going to produce the same
> build configuration as the full build.
>
Can you clarify what is going to be built with `cmake <monorepo-root>
-DLLVM_ENABLE_PROJECTS=libcxx ?

We need to cross-compile the libraries using the pre-installed clang in SDK
and have to use the standalone builds i.e. cmake <monorepo-root>/libcxx.
We
do not wish to rebuild llvm or clang when cross-compiling the libraries.
If cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=libcxx is going to build
more than libcxx, it is a *big* no-go for us (Chrome OS). Same for other
libraries like libcxxabi/libunwind.

Also,  we do not currently link libc++ with llvm's libunwind as it misses
many functionalities provided by libgcc_s/libgcc_eh in particular on ARM32
(related to GCC/GLIBC extensions) and ARM was not willing to invest in
that.
So, whilte it'd be nice to build libc++abi/libc++/libunwind in a single
innovation, linking with libunwind should NOT be mandatory.

Thanks,
Manoj

Steven
>
> >
> > -Tom
> >
> >> That also means that we can refactor some of the cmake modules to
put
> into the top level directories so sub-projects can just reuse them without
> duping the code worrying they are not available in the standalone build
> today.
> >>
> >> Steven
> >>
> >>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev <
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
> >>>
> >>> Hi folks,
> >>>
> >>> Building any LLVM project currently requires invoking CMake
inside
> <monorepo-root>/llvm, while setting the projects to enable in the
> LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing
> for the LLVM subproject happens even when one doesn't really need or
want
> it. It's also not great from a build hygiene perspective, as LLVM
globally
> sets some flags and subprojects pick them up, when they don't really
mean
> to. For example, see this workaround:
>
https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
> where we need to account for some flags that might have been set globally
> by LLVM.
> >>>
> >>> I'm not sure about other projects, however this is quite
problematic
> for projects part of the C++ runtime (libc++/libc++abi/libunwind). Indeed,
> we often try to build those projects targetting not widely supported
> platforms, where the overall LLVM build doesn't work. For example,
trying
> to use the LLVM_ENABLE_PROJECTS approach for building libc++ for
Apple's
> DriverKit environment doesn't work, since it has a few unusual things
that
> the LLVM build chokes on. However, building libc++ standalone works just
> fine because it has far fewer requirements. It's also not just an issue
of
> working vs not working: because of global flag pollution, building libc++
> standalone and as part of the rest of LLVM can result in slightly different
> flags being used, which could cause important and hard-to-diagnose issues.
> >>>
> >>> Hence, I think we should introduce a way to build LLVM
projects (or at
> least the runtimes) without going through
> <monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a
top-level
> <monorepo-root>/CMakeLists.txt whose sole job is to include
subprojects. We
> could also place basic LLVM-wide things like the check for the minimum
> CMake version there. More specifically, I would like to be able to do:
> >>>
> >>>    $ cd <monorepo-root>
> >>>    $ mkdir build
> >>>    $ (cd build && cmake <monorepo-root>
> -DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
> >>>
> >>> Pretty much the only difference with today is that you'd
use `cmake
> <monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
> >>>
> >>> Like I said, this is a problem for the runtime projects, but
I'm not
> sure about other projects. For the runtime projects, another option would
> be to only allow standalone builds. However, the runtime projects are often
> built in lockstep, and so running three CMake commands when one would
> suffice is both annoying and also an easy way to screw things up.
> Furthermore, the current standalone builds add complexity to the projects,
> because they require the ability to point to arbitrary headers/libraries
> from the other projects, when we really always want to point to the
> just-built ones.
> >>>
> >>> Relationship with Petr Hosek's "Runtimes" build
> >>>
---------------------------------------------------------------
> >>> What I'm proposing isn't a replacement for itl. The
"Runtimes" build
> can be seen as a driver that sets up the individual
> libc++/libc++abi/libunwind builds with the just-built toolchain, and for
> the provided targets. That's really great, however it is built *on top
of*
> the basic libc++/libc++abi/libunwind builds. So basically, after my
> proposal, the "Runtimes" build could simply build all elements
from the
> runtime with a single CMake invocation, as opposed to multiple invocations.
> >>>
> >>> Thoughts?
> >>> Louis
> >>>
> >>> _______________________________________________
> >>> LLVM Developers mailing list
> >>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >>
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org
> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200618/52224f39/attachment.html>

On 06/18/2020 02:40 PM, Steven Wu wrote:
> 
> 
>> On Jun 18, 2020, at 2:25 PM, Tom Stellard <tstellar at
redhat.com> wrote:
>>
>> On 06/18/2020 11:27 AM, Steven Wu via llvm-dev wrote:
>>> I like the proposal but I would like to go even further. If we are
going to create a top level CMake file, we should just go ahead and eliminate
all the standalone build configuration. The standalone build should just be
`cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=standalone-project ...`.
That means less build configuration to maintain which is always good.
>>>
>>
>> Eliminating stand-alone builds would be an inconvenience for
>> us in Fedora, since this is how we build LLVM packages.
>> However, I think we may have different definitions of what a
>> stand-alone build is.
>>
>> Does your proposal mean that
>> `cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=lld ..` would build
lld
>> only without llvm and be a replacement for what I consider to
>> be a stand-alone build, which is `cmake <monorepo-root>/lld ...`
> 
> Let me clear up the terminology here. In current llvm-project, we have a
full build and standalone build. Let say for libcxx (using libcxx or other
runtime libraries because they can build without anything from llvm
subdirectory).
> 
> This is a full build that can produce libcxx dylib:
> `cmake <monorepo-root>/llvm -DLLVM_ENABLE_PROJECTS=llvm;libcxx
-DLLVM_DISTRIBUTION_COMPONENTS=libcxx.dylib ...`
> 
> This is a standalone build that doesn't pull in anything from llvm
directory:
> `cmake <monorepo-root>/libcxx -D...`
> 
> Those two cmake configurations are going down different path and can result
in different behavior if not careful. The goal is just turning those into one
configuration. You can still do a standalone build of runtime library by
enabling just that project and it is going to produce the same build
configuration as the full build.
> 
What does libcxx currently need that is in the llvm directory?

-Tom
> Steven
> 
>>
>> -Tom
>>
>>> That also means that we can refactor some of the cmake modules to
put into the top level directories so sub-projects can just reuse them without
duping the code worrying they are not available in the standalone build today.
>>>
>>> Steven
>>>
>>>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
>>>>
>>>> Hi folks,
>>>>
>>>> Building any LLVM project currently requires invoking CMake
inside <monorepo-root>/llvm, while setting the projects to enable in the
LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing for
the LLVM subproject happens even when one doesn't really need or want it.
It's also not great from a build hygiene perspective, as LLVM globally sets
some flags and subprojects pick them up, when they don't really mean to. For
example, see this workaround:
https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
where we need to account for some flags that might have been set globally by
LLVM.
>>>>
>>>> I'm not sure about other projects, however this is quite
problematic for projects part of the C++ runtime (libc++/libc++abi/libunwind).
Indeed, we often try to build those projects targetting not widely supported
platforms, where the overall LLVM build doesn't work. For example, trying to
use the LLVM_ENABLE_PROJECTS approach for building libc++ for Apple's
DriverKit environment doesn't work, since it has a few unusual things that
the LLVM build chokes on. However, building libc++ standalone works just fine
because it has far fewer requirements. It's also not just an issue of
working vs not working: because of global flag pollution, building libc++
standalone and as part of the rest of LLVM can result in slightly different
flags being used, which could cause important and hard-to-diagnose issues.
>>>>
>>>> Hence, I think we should introduce a way to build LLVM projects
(or at least the runtimes) without going through
<monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a top-level
<monorepo-root>/CMakeLists.txt whose sole job is to include subprojects.
We could also place basic LLVM-wide things like the check for the minimum CMake
version there. More specifically, I would like to be able to do:
>>>>
>>>>    $ cd <monorepo-root>
>>>>    $ mkdir build
>>>>    $ (cd build && cmake <monorepo-root>
-DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
>>>>
>>>> Pretty much the only difference with today is that you'd
use `cmake <monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
>>>>
>>>> Like I said, this is a problem for the runtime projects, but
I'm not sure about other projects. For the runtime projects, another option
would be to only allow standalone builds. However, the runtime projects are
often built in lockstep, and so running three CMake commands when one would
suffice is both annoying and also an easy way to screw things up. Furthermore,
the current standalone builds add complexity to the projects, because they
require the ability to point to arbitrary headers/libraries from the other
projects, when we really always want to point to the just-built ones.
>>>>
>>>> Relationship with Petr Hosek's "Runtimes" build
>>>> ---------------------------------------------------------------
>>>> What I'm proposing isn't a replacement for itl. The
"Runtimes" build can be seen as a driver that sets up the individual
libc++/libc++abi/libunwind builds with the just-built toolchain, and for the
provided targets. That's really great, however it is built *on top of* the
basic libc++/libc++abi/libunwind builds. So basically, after my proposal, the
"Runtimes" build could simply build all elements from the runtime with
a single CMake invocation, as opposed to multiple invocations.
>>>>
>>>> Thoughts?
>>>> Louis
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>

> On Jun 18, 2020, at 17:25, Tom Stellard <tstellar at redhat.com>
wrote:
> 
> On 06/18/2020 11:27 AM, Steven Wu via llvm-dev wrote:
>> I like the proposal but I would like to go even further. If we are
going to create a top level CMake file, we should just go ahead and eliminate
all the standalone build configuration. The standalone build should just be
`cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=standalone-project ...`.
That means less build configuration to maintain which is always good.
>> 
> 
> Eliminating stand-alone builds would be an inconvenience for
> us in Fedora, since this is how we build LLVM packages.
> However, I think we may have different definitions of what a
> stand-alone build is.
> 
> Does your proposal mean that
> `cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=lld ..` would build lld
> only without llvm and be a replacement for what I consider to
> be a stand-alone build, which is `cmake <monorepo-root>/lld ...`
Yes, basically (if it makes sense to build lld without the rest of LLVM, which
you seem to say it does).

The difference is that it would now be clear that projects are expecting the
whole monorepo to be available in order to build, which means we can start
sharing stuff like CMake utilities & al.

Like I said, I don't know how relevant it is for other projects, but it is
really really relevant for libc++, libc++abi and libunwind. If it's not
relevant for other projects, I'm also OK with only implementing something
that works for the runtimes.

Louis
> 
> -Tom
> 
>> That also means that we can refactor some of the cmake modules to put
into the top level directories so sub-projects can just reuse them without
duping the code worrying they are not available in the standalone build today.
>> 
>> Steven
>> 
>>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
>>> 
>>> Hi folks,
>>> 
>>> Building any LLVM project currently requires invoking CMake inside
<monorepo-root>/llvm, while setting the projects to enable in the
LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing for
the LLVM subproject happens even when one doesn't really need or want it.
It's also not great from a build hygiene perspective, as LLVM globally sets
some flags and subprojects pick them up, when they don't really mean to. For
example, see this workaround:
https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
where we need to account for some flags that might have been set globally by
LLVM.
>>> 
>>> I'm not sure about other projects, however this is quite
problematic for projects part of the C++ runtime (libc++/libc++abi/libunwind).
Indeed, we often try to build those projects targetting not widely supported
platforms, where the overall LLVM build doesn't work. For example, trying to
use the LLVM_ENABLE_PROJECTS approach for building libc++ for Apple's
DriverKit environment doesn't work, since it has a few unusual things that
the LLVM build chokes on. However, building libc++ standalone works just fine
because it has far fewer requirements. It's also not just an issue of
working vs not working: because of global flag pollution, building libc++
standalone and as part of the rest of LLVM can result in slightly different
flags being used, which could cause important and hard-to-diagnose issues.
>>> 
>>> Hence, I think we should introduce a way to build LLVM projects (or
at least the runtimes) without going through
<monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a top-level
<monorepo-root>/CMakeLists.txt whose sole job is to include subprojects.
We could also place basic LLVM-wide things like the check for the minimum CMake
version there. More specifically, I would like to be able to do:
>>> 
>>>    $ cd <monorepo-root>
>>>    $ mkdir build
>>>    $ (cd build && cmake <monorepo-root>
-DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
>>> 
>>> Pretty much the only difference with today is that you'd use
`cmake <monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
>>> 
>>> Like I said, this is a problem for the runtime projects, but
I'm not sure about other projects. For the runtime projects, another option
would be to only allow standalone builds. However, the runtime projects are
often built in lockstep, and so running three CMake commands when one would
suffice is both annoying and also an easy way to screw things up. Furthermore,
the current standalone builds add complexity to the projects, because they
require the ability to point to arbitrary headers/libraries from the other
projects, when we really always want to point to the just-built ones.
>>> 
>>> Relationship with Petr Hosek's "Runtimes" build
>>> ---------------------------------------------------------------
>>> What I'm proposing isn't a replacement for itl. The
"Runtimes" build can be seen as a driver that sets up the individual
libc++/libc++abi/libunwind builds with the just-built toolchain, and for the
provided targets. That's really great, however it is built *on top of* the
basic libc++/libc++abi/libunwind builds. So basically, after my proposal, the
"Runtimes" build could simply build all elements from the runtime with
a single CMake invocation, as opposed to multiple invocations.
>>> 
>>> Thoughts?
>>> Louis
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>