llvm dev - Nov 2015 - [RFC] Strategies for Bootstrapping Compiler-RT builtins

> On Nov 2, 2015, at 11:41 AM, Vasileios Kalintiris <Vasileios.Kalintiris
at imgtec.com> wrote:
> 
>> The problem comes when bootstrapping a cross-compiler toolchain. In
order to
>> have a cross-compiling toolchain that can build a “hello world”
application you
>> need four basic components:
>> 
>> (1) clang
>> (2) ld
>> (3) libclang_rt (builtins)
>> (4) runtime libraries
>> 
>> Today building this toolchain with CMake is impossible because you
cannot
>> configure the cross-compiled builtins. The failure is a result of
CMake’s
>> try_compile function always testing a full compile then link operation.
When
>> bootstrapping a cross-compiler this will always fail because linking
even the
>> simplest applications fails when you don’t have libclang_rt prebuilt.
> 
> In my case, I had to use the CMAKE_{C,CXX}_COMPILER_FORCED variables and
> guestimate the appropriate C/C++ flags in order to avoid CMake getting in
the
> way with the various checks. My scripts build clang & lld, then install
the
> linux's kernel & Musl's C library headers and finally build
compiler-rt's
> builtins library (by setting COMPILER_RT_BUILD_SANITIZERS=Off). The next
steps
> build Musl, libunwind, libcxx and libcxxabi. This way I'm able to
"use" CMake
> for every LLVM project.
Setting the *_FORCED variables is roughly equivalent to setting the *_WORKS
variables. I’d really like it if we didn’t need to set those.
> 
> What isn't clear to me is whether you intend to extend your work to
libunwind,
> libcxx & libcxxabi because they do require a working C++ compiler too.
Also,
> my understanding is that the same is true for the sanitizers libraries. 
I have a separate set of patches I’m working on now to generalize our use of
ExternalProject so we can build libcxx, libcxxabi, libunwind, and anything else
that comes along with just-built compilers to fix this.
> 
>> So, how do we fix this? I have a couple ideas, and am open to more.
>> 
>> ...
>> 
>> (3) Split the builtins and the runtime libraries
>> 
>> This is the most complicated approach, but I also think it is the best
>> approach. One of the underlying problems here is that the builtin
libraries
>> and the runtime libraries have very different requirements for
building. The
>> builtins really only require a functional compiler and archiver, and
the
>> runtime libraries require a full linker + runtime libraries (libc &
libcxx).
>> These additional build-time requirements actually make things very
>> complicated because when bootstrapping a cross toolchain compiler-rt
needs to
>> build in two different places in the build order; once before libcxx,
and
>> once after.
> 
> IMHO, from the options that you've mentioned, (3) is the cleanest one. 
> Conceptually, as you mentioned the builtins library does not belong with
the
> sanitizers and it would make sense splitting them in separate repositories.
> Would it be too difficult to create an initial split given that we already
> provide/support the COMPILER_RT_BUILD_BUILTINS and
COMPILER_RT_BUILD_SANITIZERS
> options?
I think there is a lot of work involved here. We’ll need to generalize and split
up some of the configuration content (stuff like config-ix.cmake) into the bits
that apply everywhere, the bits that are needed only for builtins, and the bits
that are only needed for sanitizers. I’m willing to start tackling that work if
this is what the community thinks the right approach is.
> 
>> Either way, supporting this approach will require significant cleanup
and
>> refactoring because we’ll need to separate out the build system
functionality
>> into three categories: things that apply to builtins, things that apply
to
>> runtimes, things that apply to both.
> 
> I don't have enought experience with CMake so I don't know how
feasible this is,
> but for the last two categories it would be nice to keep in mind that the
user
> might have a C library other than Glibc.
I’m certainly not going to expect the user has Glibc. I do all my work on Darwin
platforms and we don’t have Glibc.

-Chris
> 
> - Vasileios

> Setting the *_FORCED variables is roughly equivalent to setting the *_WORKS
variables. I’d really like it if we didn’t need to set those.
Yes, I believe the *_FORCED variables are a little bit more intrusive than the
*_WORKS variables.
>> IMHO, from the options that you've mentioned, (3) is the cleanest
one.
>> Conceptually, as you mentioned the builtins library does not belong
with the
>> sanitizers and it would make sense splitting them in separate
repositories.
>> Would it be too difficult to create an initial split given that we
already
>> provide/support the COMPILER_RT_BUILD_BUILTINS and
COMPILER_RT_BUILD_SANITIZERS
>> options?
>
> I think there is a lot of work involved here. We’ll need to generalize and
split up
> some of the configuration content (stuff like config-ix.cmake) into the
bits that apply
> everywhere, the bits that are needed only for builtins, and the bits that
are only
> needed for sanitizers. I’m willing to start tackling that work if this is
what
> the community thinks the right approach is.
>
Uh, you're right. I forgot about the common checks in config-ix.cmake.
>> I don't have enought experience with CMake so I don't know how
feasible this is,
>> but for the last two categories it would be nice to keep in mind that
the user
>> might have a C library other than Glibc.
>
> I’m certainly not going to expect the user has Glibc. I do all my work on
Darwin platforms and we don’t have Glibc.
What I meant here is that some user might want to use another C library, other
than the one installed in the system.
This would require us to be able to build the C library in between the LLVM
projects, ie. after builtins. This
is just a suggestion as I don't know how much work/effort this would take
(or if it requires any work at all).

- Vasileios

> On Nov 2, 2015, at 1:45 PM, Vasileios Kalintiris <Vasileios.Kalintiris
at imgtec.com> wrote:
> 
>> Setting the *_FORCED variables is roughly equivalent to setting the
*_WORKS variables. I’d really like it if we didn’t need to set those.
> 
> Yes, I believe the *_FORCED variables are a little bit more intrusive than
the *_WORKS variables.
> 
>>> IMHO, from the options that you've mentioned, (3) is the
cleanest one.
>>> Conceptually, as you mentioned the builtins library does not belong
with the
>>> sanitizers and it would make sense splitting them in separate
repositories.
>>> Would it be too difficult to create an initial split given that we
already
>>> provide/support the COMPILER_RT_BUILD_BUILTINS and
COMPILER_RT_BUILD_SANITIZERS
>>> options?
>> 
>> I think there is a lot of work involved here. We’ll need to generalize
and split up
>> some of the configuration content (stuff like config-ix.cmake) into the
bits that apply
>> everywhere, the bits that are needed only for builtins, and the bits
that are only
>> needed for sanitizers. I’m willing to start tackling that work if this
is what
>> the community thinks the right approach is.
>> 
> 
> Uh, you're right. I forgot about the common checks in config-ix.cmake.
> 
>>> I don't have enought experience with CMake so I don't know
how feasible this is,
>>> but for the last two categories it would be nice to keep in mind
that the user
>>> might have a C library other than Glibc.
>> 
>> I’m certainly not going to expect the user has Glibc. I do all my work
on Darwin platforms and we don’t have Glibc.
> 
> What I meant here is that some user might want to use another C library,
other than the one installed in the system.
> This would require us to be able to build the C library in between the LLVM
projects, ie. after builtins. This
> is just a suggestion as I don't know how much work/effort this would
take (or if it requires any work at all).
Sure. I’m not sure if everyone will agree with me, but my general assertion is
that if you’re doing a full bootstrap of a cross compiler, we’re not going to
fully support that from a single CMake build tree. We didn’t support that in
autoconf, so I don’t think CMake should be different in that regard. I envision
bootstrapping a full cross-platform to be something like:

(1) build in-tree clang
(2) build out-of-tree builtin library
(3) build out-of-tree runtime libraries

-Chris
> 
> - Vasileios