llvm dev - Jun 2015 - [LLVMdev] [RFC] Ideas on improving Compiler-RT CMake

LLVMDev,

PR 15732 is the umbrella tracking the progress of making the CMake build system
feature equivalent to the autotools one. One of the biggest outstanding tasks is
PR 21562, which I've been viewing more as a "We really need to fix how
we build Compiler-RT."

I've thought about this quite a bit, and I wanted to send out some of my
ideas to get some feedback and have some discussions about the way forward.

The complication with Compiler-RT is that it is inherently a cross-compile. When
you build LLVM, Clang and Compiler-RT, you really want to build LLVM, Clang, and
n Compiler-RTs where n is the valid combinations of target architectures and/or
platforms your clang supports.

There are three basic configurations for building Compiler-RT.

(1) Building a "pure" Clang supporting targeting your host
(2) Building a cross-targeting Clang that runs on one OS or architecture and
targets another
(3) Building a Clang that both self-targets and cross-targets

The overall change I want to make to Compiler-RT is that I want to handle case 1
as an in-tree build similar to how it is today, case 2 as a cross-compile, and
case 3 as a combination of cases 1 & 2.

Let me walk through an example of how I think this should work.

If my host is OS X and I'm building Clang to target OS X and iOS, when I
configure my build directory there will be an in-tree build of Compiler-RT (via
add_llvm_external_project) to generate the OS X runtime libraries, and an
out-of-tree-style cross-compile generated for building the iOS runtime libraries
for all supported architectures. This would all be driven via command line
settings like:

LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"

This would generate universal binary outputs in the format:
libclangrt_<lib>.<os>.<ext>

As a second example, if you were on Windows building a Clang to target only
FreeBSD when you configured your build directory there would only be an
out-of-tree-style cross-compile generated for your target architecture against
FreeBSD. The command line setting here would be:

LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"

This would generate binary outputs in the format:
libclangrt_<lib>.<arch>.<ext>

Similarly if you were on linux targeting linux you’d only get an in-tree build
with outputs:
libclangrt_<lib>.<arch>.<ext>

The cross-compile builds will be driven via CMake's toolchain file settings,
so if you were cross-targeting and need to set compiler options like -isysroot
there will need to be a toolchain file defined for your target platform.

Using toolchain files and cross-compile configurations solves one of the
problems with our current CMake system where the Compiler-RT builds don't
re-run the CMake checks. Not re-running the CMake checks causes significant
problems if your host and target OS are different.

So, how do we get to this state?

Honestly I don't have a complete plan, but I have some parts of a plan.

The first thing I'd like to do is try to make a bunch of the Darwin-specific
code go away. Obviously there will always need to be some at a lower level
because Darwin is different, but some of the lower-level functions like
add_compiler_rt_darwin_object_library probably don't need to exist.

One example of the type of thing I want to get rid of is the ASan
CMakeLists.txt. The top-level CMake files for libraries and executables
shouldn't need to care about the targets. Today we loop over OSs for darwin
and call add_compiler_rt_darwin_object_library, and for other operating systems
we loop over archs and call add_compiler_rt_object_library. I want to push the
looping as low down into the utility functions as possible. Doing this will
reduce the amount of Darwin-specific code because we replicate this exact
looping behavior all over Compiler-RT's build.

The second thing I want to do is make the Darwin build able to generate binaries
for just OSX or just iOS. Today the Darwin build tries to build everything all
the time, and that really isn't the behavior we need if we're going to
treat iOS as a cross-compile.

The third step is to make the Darwin llvm/clang/compiler-rt combined build use
cross-compilation for building the iOS runtime libraries. I want to do this on
Darwin only because that's the platform I'm most familiar with. At some
point if we want to move other targets to cross-compiling I will need help from
other people in the community who are more familiar with the specific targets.

With each step I'll be trying to remove more OS-specific code and trying to
streamline building.

Questions/Comments/Concerns/Cries of insanity?

Thanks,
-Chris

Looping in cfe-dev because I should have send this there too from the start.

-Chris
> On Jun 1, 2015, at 3:21 PM, Chris Bieneman <beanz at apple.com>
wrote:
> 
> LLVMDev,
> 
> PR 15732 is the umbrella tracking the progress of making the CMake build
system feature equivalent to the autotools one. One of the biggest outstanding
tasks is PR 21562, which I've been viewing more as a "We really need to
fix how we build Compiler-RT."
> 
> I've thought about this quite a bit, and I wanted to send out some of
my ideas to get some feedback and have some discussions about the way forward.
> 
> The complication with Compiler-RT is that it is inherently a cross-compile.
When you build LLVM, Clang and Compiler-RT, you really want to build LLVM,
Clang, and n Compiler-RTs where n is the valid combinations of target
architectures and/or platforms your clang supports.
> 
> There are three basic configurations for building Compiler-RT.
> 
> (1) Building a "pure" Clang supporting targeting your host
> (2) Building a cross-targeting Clang that runs on one OS or architecture
and targets another
> (3) Building a Clang that both self-targets and cross-targets
> 
> The overall change I want to make to Compiler-RT is that I want to handle
case 1 as an in-tree build similar to how it is today, case 2 as a
cross-compile, and case 3 as a combination of cases 1 & 2.
> 
> Let me walk through an example of how I think this should work.
> 
> If my host is OS X and I'm building Clang to target OS X and iOS, when
I configure my build directory there will be an in-tree build of Compiler-RT
(via add_llvm_external_project) to generate the OS X runtime libraries, and an
out-of-tree-style cross-compile generated for building the iOS runtime libraries
for all supported architectures. This would all be driven via command line
settings like:
> 
> LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
> LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"
> 
> This would generate universal binary outputs in the format:
> libclangrt_<lib>.<os>.<ext>
> 
> As a second example, if you were on Windows building a Clang to target only
FreeBSD when you configured your build directory there would only be an
out-of-tree-style cross-compile generated for your target architecture against
FreeBSD. The command line setting here would be:
> 
> LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"
> 
> This would generate binary outputs in the format:
> libclangrt_<lib>.<arch>.<ext>
> 
> Similarly if you were on linux targeting linux you’d only get an in-tree
build with outputs:
> libclangrt_<lib>.<arch>.<ext>
> 
> The cross-compile builds will be driven via CMake's toolchain file
settings, so if you were cross-targeting and need to set compiler options like
-isysroot there will need to be a toolchain file defined for your target
platform.
> 
> Using toolchain files and cross-compile configurations solves one of the
problems with our current CMake system where the Compiler-RT builds don't
re-run the CMake checks. Not re-running the CMake checks causes significant
problems if your host and target OS are different.
> 
> So, how do we get to this state?
> 
> Honestly I don't have a complete plan, but I have some parts of a plan.
> 
> The first thing I'd like to do is try to make a bunch of the
Darwin-specific code go away. Obviously there will always need to be some at a
lower level because Darwin is different, but some of the lower-level functions
like add_compiler_rt_darwin_object_library probably don't need to exist.
> 
> One example of the type of thing I want to get rid of is the ASan
CMakeLists.txt. The top-level CMake files for libraries and executables
shouldn't need to care about the targets. Today we loop over OSs for darwin
and call add_compiler_rt_darwin_object_library, and for other operating systems
we loop over archs and call add_compiler_rt_object_library. I want to push the
looping as low down into the utility functions as possible. Doing this will
reduce the amount of Darwin-specific code because we replicate this exact
looping behavior all over Compiler-RT's build.
> 
> The second thing I want to do is make the Darwin build able to generate
binaries for just OSX or just iOS. Today the Darwin build tries to build
everything all the time, and that really isn't the behavior we need if
we're going to treat iOS as a cross-compile.
> 
> The third step is to make the Darwin llvm/clang/compiler-rt combined build
use cross-compilation for building the iOS runtime libraries. I want to do this
on Darwin only because that's the platform I'm most familiar with. At
some point if we want to move other targets to cross-compiling I will need help
from other people in the community who are more familiar with the specific
targets.
> 
> With each step I'll be trying to remove more OS-specific code and
trying to streamline building.
> 
> Questions/Comments/Concerns/Cries of insanity?
> 
> Thanks,
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On 6/1/15 6:42 PM, Chris Bieneman wrote:
> Looping in cfe-dev because I should have send this there too from the
> start.
>
> -Chris
>
>> On Jun 1, 2015, at 3:21 PM, Chris Bieneman <beanz at apple.com>
>> wrote:
>>
>> LLVMDev,
>>
>> PR 15732 is the umbrella tracking the progress of making the CMake
>> build system feature equivalent to the autotools one. One of the
>> biggest outstanding tasks is PR 21562, which I've been viewing more
>> as a "We really need to fix how we build Compiler-RT."
+1
>>
>> I've thought about this quite a bit, and I wanted to send out some
>> of my ideas to get some feedback and have some discussions about
>> the way forward.
>>
>> The complication with Compiler-RT is that it is inherently a
>> cross-compile. When you build LLVM, Clang and Compiler-RT, you
>> really want to build LLVM, Clang, and n Compiler-RTs where n is the
>> valid combinations of target architectures and/or platforms your
>> clang supports.
>>
>> There are three basic configurations for building Compiler-RT.
>>
>> (1) Building a "pure" Clang supporting targeting your host
(2)
>> Building a cross-targeting Clang that runs on one OS or
>> architecture and targets another (3) Building a Clang that both
>> self-targets and cross-targets
>>
>> The overall change I want to make to Compiler-RT is that I want to
>> handle case 1 as an in-tree build similar to how it is today, case
>> 2 as a cross-compile, and case 3 as a combination of cases 1 & 2.
Why treat $host as a special case? IM-not-so-humble-O, $host is just 
another $target.

I think the ideal case is to have a table of:

   (triple, compiler, flags, sysroot)

tuples that say which configurations of the library to build.

If Clang is compiler-rt's compiler, then all of the compiler entries in 
that table can point to the same binary. For in-tree builds, we *should* 
be using the "just built" Clang anyway (but are not currently, without
enabling the flag to do so). Either way, I think we ought to move away 
from thinking that a native build is in some way "special".

If GCC is compiler-rt's compiler, then there would have to have a 
different binary per triple.

If we drop support for building compiler-rt with GCC, this gets even 
simpler. Compiler-rt is *Clang's* runtime library, after all.

The plus side of treating *every* build as a cross build is that then 
there will be less risk of builds differing between native builds and 
cross builds when the target is the same (for example, linux targeting 
linux vs darwin targeting linux).
>>
>> Let me walk through an example of how I think this should work.
>>
>> If my host is OS X and I'm building Clang to target OS X and iOS,
>> when I configure my build directory there will be an in-tree build
>> of Compiler-RT (via add_llvm_external_project) to generate the OS X
>> runtime libraries, and an out-of-tree-style cross-compile generated
>> for building the iOS runtime libraries for all supported
>> architectures. This would all be driven via command line settings
>> like:
>>
>> LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
>> LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"
>>
>> This would generate universal binary outputs in the format:
>> libclangrt_<lib>.<os>.<ext>
>>
>> As a second example, if you were on Windows building a Clang to
>> target only FreeBSD when you configured your build directory there
>> would only be an out-of-tree-style cross-compile generated for your
>> target architecture against FreeBSD. The command line setting here
>> would be:
>>
>> LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"
>>
>> This would generate binary outputs in the format:
>> libclangrt_<lib>.<arch>.<ext>
>>
>> Similarly if you were on linux targeting linux you’d only get an
>> in-tree build with outputs:
libclangrt_<lib>.<arch>.<ext>
>>
>> The cross-compile builds will be driven via CMake's toolchain file
>> settings, so if you were cross-targeting and need to set compiler
>> options like -isysroot there will need to be a toolchain file
>> defined for your target platform.
>>
>> Using toolchain files and cross-compile configurations solves one
>> of the problems with our current CMake system where the Compiler-RT
>> builds don't re-run the CMake checks. Not re-running the CMake
>> checks causes significant problems if your host and target OS are
>> different.
Can you elaborate on that last sentence? I'm not very familiar with 
CMake, and that's a bit of a surprise to me.
>>
>> So, how do we get to this state?
>>
>> Honestly I don't have a complete plan, but I have some parts of a
>> plan.
>>
>> The first thing I'd like to do is try to make a bunch of the
>> Darwin-specific code go away. Obviously there will always need to
>> be some at a lower level because Darwin is different, but some of
>> the lower-level functions like
>> add_compiler_rt_darwin_object_library probably don't need to
>> exist.
>>
>> One example of the type of thing I want to get rid of is the ASan
>> CMakeLists.txt. The top-level CMake files for libraries and
>> executables shouldn't need to care about the targets. Today we loop
>> over OSs for darwin and call add_compiler_rt_darwin_object_library,
>> and for other operating systems we loop over archs and call
>> add_compiler_rt_object_library. I want to push the looping as low
>> down into the utility functions as possible. Doing this will reduce
>> the amount of Darwin-specific code because we replicate this exact
>> looping behavior all over Compiler-RT's build.
>>
>> The second thing I want to do is make the Darwin build able to
>> generate binaries for just OSX or just iOS. Today the Darwin build
>> tries to build everything all the time, and that really isn't the
>> behavior we need if we're going to treat iOS as a cross-compile.
>>
>> The third step is to make the Darwin llvm/clang/compiler-rt
>> combined build use cross-compilation for building the iOS runtime
>> libraries. I want to do this on Darwin only because that's the
>> platform I'm most familiar with. At some point if we want to move
>> other targets to cross-compiling I will need help from other people
>> in the community who are more familiar with the specific targets.
>>
>> With each step I'll be trying to remove more OS-specific code and
>> trying to streamline building.
>>
>> Questions/Comments/Concerns/Cries of insanity?
Thanks for taking the driver's seat on the discussions of all this CMake 
transition stuff, BTW.


Cheers,

Jon
>>
>> Thanks, -Chris _______________________________________________ LLVM
>> Developers mailing list LLVMdev at cs.uiuc.edu
>> http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
> _______________________________________________ LLVM Developers
> mailing list LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded

-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded

I'm actually pretty leery of using the CMake cross-compiling support
because it requires using multiple build directories with split build
systems. We end up a situation where the parent build shells out to cmake
and then recursively reinvokes the parent build system on the new build
directory. Running cmake again in the first place is really slow (lots of
test compiles in serial), and recursively invoking the build system leads
to really bad job scheduling on all build systems other than 'make'. In
particular, ninja shelling out to ninja will oversaturate the system, and I
don't expect it to ever add job server functionality like make has.

How much are we really getting from using the upstream cmake
cross-compiling support? Most of the header and symbol checks that we have
are for the sanitizer test suite, which I doubt needs generalized
cross-compilation support. Only the <unwind.h> check appears to matter for
lib/builtins. Maybe we should roll our own custom rules for invoking the
just-built clang compiler like we already do for the sanitizer test suite?
I cc'd Alexey, who I believe wrote that support.

Anyway, thanks for tackling this, you're the one doing the work, and I'm
not volunteering. :) Do what you think is best. I mostly wanted to present
a possible alternative approach.

On Mon, Jun 1, 2015 at 3:21 PM, Chris Bieneman <beanz at apple.com> wrote:
> LLVMDev,
>
> PR 15732 is the umbrella tracking the progress of making the CMake build
> system feature equivalent to the autotools one. One of the biggest
> outstanding tasks is PR 21562, which I've been viewing more as a
"We really
> need to fix how we build Compiler-RT."
>
> I've thought about this quite a bit, and I wanted to send out some of
my
> ideas to get some feedback and have some discussions about the way forward.
>
> The complication with Compiler-RT is that it is inherently a
> cross-compile. When you build LLVM, Clang and Compiler-RT, you really want
> to build LLVM, Clang, and n Compiler-RTs where n is the valid combinations
> of target architectures and/or platforms your clang supports.
>
> There are three basic configurations for building Compiler-RT.
>
> (1) Building a "pure" Clang supporting targeting your host
> (2) Building a cross-targeting Clang that runs on one OS or architecture
> and targets another
> (3) Building a Clang that both self-targets and cross-targets
>
> The overall change I want to make to Compiler-RT is that I want to handle
> case 1 as an in-tree build similar to how it is today, case 2 as a
> cross-compile, and case 3 as a combination of cases 1 & 2.
>
> Let me walk through an example of how I think this should work.
>
> If my host is OS X and I'm building Clang to target OS X and iOS, when
I
> configure my build directory there will be an in-tree build of Compiler-RT
> (via add_llvm_external_project) to generate the OS X runtime libraries, and
> an out-of-tree-style cross-compile generated for building the iOS runtime
> libraries for all supported architectures. This would all be driven via
> command line settings like:
>
> LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
> LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"
>
> This would generate universal binary outputs in the format:
> libclangrt_<lib>.<os>.<ext>
>
> As a second example, if you were on Windows building a Clang to target
> only FreeBSD when you configured your build directory there would only be
> an out-of-tree-style cross-compile generated for your target architecture
> against FreeBSD. The command line setting here would be:
>
> LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"
>
> This would generate binary outputs in the format:
> libclangrt_<lib>.<arch>.<ext>
>
> Similarly if you were on linux targeting linux you’d only get an in-tree
> build with outputs:
> libclangrt_<lib>.<arch>.<ext>
>
> The cross-compile builds will be driven via CMake's toolchain file
> settings, so if you were cross-targeting and need to set compiler options
> like -isysroot there will need to be a toolchain file defined for your
> target platform.
>
> Using toolchain files and cross-compile configurations solves one of the
> problems with our current CMake system where the Compiler-RT builds
don't
> re-run the CMake checks. Not re-running the CMake checks causes significant
> problems if your host and target OS are different.
>
> So, how do we get to this state?
>
> Honestly I don't have a complete plan, but I have some parts of a plan.
>
> The first thing I'd like to do is try to make a bunch of the
> Darwin-specific code go away. Obviously there will always need to be some
> at a lower level because Darwin is different, but some of the lower-level
> functions like add_compiler_rt_darwin_object_library probably don't
need to
> exist.
>
> One example of the type of thing I want to get rid of is the ASan
> CMakeLists.txt. The top-level CMake files for libraries and executables
> shouldn't need to care about the targets. Today we loop over OSs for
darwin
> and call add_compiler_rt_darwin_object_library, and for other operating
> systems we loop over archs and call add_compiler_rt_object_library. I want
> to push the looping as low down into the utility functions as possible.
> Doing this will reduce the amount of Darwin-specific code because we
> replicate this exact looping behavior all over Compiler-RT's build.
>
> The second thing I want to do is make the Darwin build able to generate
> binaries for just OSX or just iOS. Today the Darwin build tries to build
> everything all the time, and that really isn't the behavior we need if
> we're going to treat iOS as a cross-compile.
>
> The third step is to make the Darwin llvm/clang/compiler-rt combined build
> use cross-compilation for building the iOS runtime libraries. I want to do
> this on Darwin only because that's the platform I'm most familiar
with. At
> some point if we want to move other targets to cross-compiling I will need
> help from other people in the community who are more familiar with the
> specific targets.
>
> With each step I'll be trying to remove more OS-specific code and
trying
> to streamline building.
>
> Questions/Comments/Concerns/Cries of insanity?
>
> Thanks,
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150604/e1ca1a83/attachment.html>

This is definitely a concern. That’s why I had proposed not treating “host” as a
cross-target. Handling building compiler-rt for host differently makes one of
the common cases better.

WRT Ninja.

Ninja’s pooling capabilities actually should solve some of the problems you’re
voicing. In particular, we can make the cross-builds fall into the “console”
pool to prevent ninja from over-saturating the system
(https://martine.github.io/ninja/manual.html#_the_literal_console_literal_pool
<https://martine.github.io/ninja/manual.html#_the_literal_console_literal_pool>).

I also expect that we could engage with the CMake community to come up with
better tooling for cross-compiling to help mitigate some of this. In particular
I would really love to have a way import a configured project into the project
you’re configuring, and have it generate a single build configuration that
doesn’t just call out to CMake to build the other project.

I honestly don’t know a whole lot about the compiler-rt codebase. I’ve stepped
into this project largely just as an extension of my hatred for autoconf. I
don’t know how much we’ll get from CMake’s cross-compilation, but I do know that
the way we handle cross-targeting CMake today for iOS leaves a lot to be
desired.

One of my concerns is actually that the custom rules we have for the sanitizer
test suite doesn’t work at all on Darwin unless you have the OS headers
installed to / (which isn’t the default anymore). I think that teaching CMake to
be smart enough about how it invokes the compiler to be able to support all our
platforms will be a lot of work duplicating things CMake itself already does.

-Chris

> On Jun 4, 2015, at 2:25 PM, Reid Kleckner <rnk at google.com> wrote:
> 
> I'm actually pretty leery of using the CMake cross-compiling support
because it requires using multiple build directories with split build systems.
We end up a situation where the parent build shells out to cmake and then
recursively reinvokes the parent build system on the new build directory.
Running cmake again in the first place is really slow (lots of test compiles in
serial), and recursively invoking the build system leads to really bad job
scheduling on all build systems other than 'make'. In particular, ninja
shelling out to ninja will oversaturate the system, and I don't expect it to
ever add job server functionality like make has.
> 
> How much are we really getting from using the upstream cmake
cross-compiling support? Most of the header and symbol checks that we have are
for the sanitizer test suite, which I doubt needs generalized cross-compilation
support. Only the <unwind.h> check appears to matter for lib/builtins.
Maybe we should roll our own custom rules for invoking the just-built clang
compiler like we already do for the sanitizer test suite? I cc'd Alexey, who
I believe wrote that support.
> 
> Anyway, thanks for tackling this, you're the one doing the work, and
I'm not volunteering. :) Do what you think is best. I mostly wanted to
present a possible alternative approach.
> 
> On Mon, Jun 1, 2015 at 3:21 PM, Chris Bieneman <beanz at apple.com
<mailto:beanz at apple.com>> wrote:
> LLVMDev,
> 
> PR 15732 is the umbrella tracking the progress of making the CMake build
system feature equivalent to the autotools one. One of the biggest outstanding
tasks is PR 21562, which I've been viewing more as a "We really need to
fix how we build Compiler-RT."
> 
> I've thought about this quite a bit, and I wanted to send out some of
my ideas to get some feedback and have some discussions about the way forward.
> 
> The complication with Compiler-RT is that it is inherently a cross-compile.
When you build LLVM, Clang and Compiler-RT, you really want to build LLVM,
Clang, and n Compiler-RTs where n is the valid combinations of target
architectures and/or platforms your clang supports.
> 
> There are three basic configurations for building Compiler-RT.
> 
> (1) Building a "pure" Clang supporting targeting your host
> (2) Building a cross-targeting Clang that runs on one OS or architecture
and targets another
> (3) Building a Clang that both self-targets and cross-targets
> 
> The overall change I want to make to Compiler-RT is that I want to handle
case 1 as an in-tree build similar to how it is today, case 2 as a
cross-compile, and case 3 as a combination of cases 1 & 2.
> 
> Let me walk through an example of how I think this should work.
> 
> If my host is OS X and I'm building Clang to target OS X and iOS, when
I configure my build directory there will be an in-tree build of Compiler-RT
(via add_llvm_external_project) to generate the OS X runtime libraries, and an
out-of-tree-style cross-compile generated for building the iOS runtime libraries
for all supported architectures. This would all be driven via command line
settings like:
> 
> LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
> LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"
> 
> This would generate universal binary outputs in the format:
> libclangrt_<lib>.<os>.<ext>
> 
> As a second example, if you were on Windows building a Clang to target only
FreeBSD when you configured your build directory there would only be an
out-of-tree-style cross-compile generated for your target architecture against
FreeBSD. The command line setting here would be:
> 
> LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"
> 
> This would generate binary outputs in the format:
> libclangrt_<lib>.<arch>.<ext>
> 
> Similarly if you were on linux targeting linux you’d only get an in-tree
build with outputs:
> libclangrt_<lib>.<arch>.<ext>
> 
> The cross-compile builds will be driven via CMake's toolchain file
settings, so if you were cross-targeting and need to set compiler options like
-isysroot there will need to be a toolchain file defined for your target
platform.
> 
> Using toolchain files and cross-compile configurations solves one of the
problems with our current CMake system where the Compiler-RT builds don't
re-run the CMake checks. Not re-running the CMake checks causes significant
problems if your host and target OS are different.
> 
> So, how do we get to this state?
> 
> Honestly I don't have a complete plan, but I have some parts of a plan.
> 
> The first thing I'd like to do is try to make a bunch of the
Darwin-specific code go away. Obviously there will always need to be some at a
lower level because Darwin is different, but some of the lower-level functions
like add_compiler_rt_darwin_object_library probably don't need to exist.
> 
> One example of the type of thing I want to get rid of is the ASan
CMakeLists.txt. The top-level CMake files for libraries and executables
shouldn't need to care about the targets. Today we loop over OSs for darwin
and call add_compiler_rt_darwin_object_library, and for other operating systems
we loop over archs and call add_compiler_rt_object_library. I want to push the
looping as low down into the utility functions as possible. Doing this will
reduce the amount of Darwin-specific code because we replicate this exact
looping behavior all over Compiler-RT's build.
> 
> The second thing I want to do is make the Darwin build able to generate
binaries for just OSX or just iOS. Today the Darwin build tries to build
everything all the time, and that really isn't the behavior we need if
we're going to treat iOS as a cross-compile.
> 
> The third step is to make the Darwin llvm/clang/compiler-rt combined build
use cross-compilation for building the iOS runtime libraries. I want to do this
on Darwin only because that's the platform I'm most familiar with. At
some point if we want to move other targets to cross-compiling I will need help
from other people in the community who are more familiar with the specific
targets.
> 
> With each step I'll be trying to remove more OS-specific code and
trying to streamline building.
> 
> Questions/Comments/Concerns/Cries of insanity?
> 
> Thanks,
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>        
http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/>
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
<http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150604/c544c7f8/attachment.html>