llvm dev - Jun 2016 - [RFC] LLVM Directory Structure Changes (was Re: [PATCH] D20992: [CMake] Add LLVM runtimes directory)

On 12 June 2016 at 00:35, Sean Silva via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>> I think the fundamental distinction needs to be following dependency
>> graphs because if we don’t get rid of the circular dependency in
>> bootstrapping there is no reason to make any changes.
>
> +1 for what Chandler said here.
>
> I don't think CMake itself per se cares about actual VCS repo
breakdown. It
> should be possible to bring sanity without any change to VCS structure.
Funny enough "compiler-rt" is a fitting name to anything that provides
run-time libraries that users shouldn't be fiddling with, and that
includes builtins, sanitizers, profiling, the lot.

Not getting into the name and split bike shed, I agree with Sean that
this is a CMake issue, not a directory structure one.

Simply put, RT's CMake is a mess.

If we can build each component (builtins, sans, profs) independently,
then the overall build dependency can follow without a split, by
basically creating internal independent targets with a phony target
that builds them all being the default.

So, if in the main CMake you ask for libc++, RT and unwind, the build
dependency will be from specific internal targets in libc++ to other
internal targets in RT and so on.

This could also be a way out to separate the "builtins" target into
"builtins.<arch>" into different internal targets to allow
cross-compilation. Though, that'd requiring having a complete
toolchain for each of the supported target, so it can't be default on
the targets built in LLVM.

cheers,
--renato

I think I may be explaining the dependency problem poorly. This isn't so
much a limitation in CMake, or a problem with our CMake implementation. This is
a limitation in basically every build configuration management system I've
ever seen.

The problem comes from that fact that generally builds are structured as
configure -> build. What we actually need is "configure some" ->
"build some" -> "configure some more" -> "build
some more" -> repeat until done.

The problem occurs when you are bootstrapping a toolchain and target. In general
a bootstrap build involves building a compiler for your target (which may or may
not match your host), then building the various libraries and tools, working up
to the full OS.

For our projects the first step of a bootstrap is building a clang that runs on
host supporting the target. After building that clang you need to build the
runtime libraries. For sake of simplicity let's assume we're only
building compiler-rt's current libraries. After building clang the logical
step would be CMake + Ninja for compiler-rt then you're done right? Except,
that doesn't work. You can't *configure* the sanitizer build until after
you've fully configured and built the builtin libraries.

Pardon the bad formatting here, but this next sentence is really important, so
I'm going to try and apply some emphasis.

***Generally speaking, you cannot *configure* a project until all the tools and
libraries your build system is running tests against are built.***

If you configure a project before the tools and libraries are available your
configuration system may make the wrong decisions. To put it in more normal
compiler terminology, it is undefined behavior.

We have viable workarounds for this in our build today. We can extend these
workarounds to make bootstrapping simpler. There are some rough edges that are
caused by limitations in CMake, but there is no fundamental limitation in CMake
that makes this not work.

At Apple we workaround this by having some out-of-tree script goop that
configures and builds clang, then configures and builds the builtins, then
configures and builds the runtimes. Due to complications interfacing with our
internal build system, we'll likely always have that script goop, but
I'd really like to be able to run clean bootstrap builds against open source
without the crazy complications.

I believe that splitting compiler-rt's sanitizer libraries and builtin
archives into separate projects will allow the project structure to more clearly
represent the way it fits into the project build graph. It is not strictly
necessary for anything, so if people generally don't agree I won't fight
for it.

-Chris
> On Jun 12, 2016, at 4:51 AM, Renato Golin <renato.golin at
linaro.org> wrote:
> 
> On 12 June 2016 at 00:35, Sean Silva via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>> I think the fundamental distinction needs to be following
dependency
>>> graphs because if we don’t get rid of the circular dependency in
>>> bootstrapping there is no reason to make any changes.
>> 
>> +1 for what Chandler said here.
>> 
>> I don't think CMake itself per se cares about actual VCS repo
breakdown. It
>> should be possible to bring sanity without any change to VCS structure.
> 
> Funny enough "compiler-rt" is a fitting name to anything that
provides
> run-time libraries that users shouldn't be fiddling with, and that
> includes builtins, sanitizers, profiling, the lot.
> 
> Not getting into the name and split bike shed, I agree with Sean that
> this is a CMake issue, not a directory structure one.
> 
> Simply put, RT's CMake is a mess.
> 
> If we can build each component (builtins, sans, profs) independently,
> then the overall build dependency can follow without a split, by
> basically creating internal independent targets with a phony target
> that builds them all being the default.
> 
> So, if in the main CMake you ask for libc++, RT and unwind, the build
> dependency will be from specific internal targets in libc++ to other
> internal targets in RT and so on.
> 
> This could also be a way out to separate the "builtins" target
into
> "builtins.<arch>" into different internal targets to allow
> cross-compilation. Though, that'd requiring having a complete
> toolchain for each of the supported target, so it can't be default on
> the targets built in LLVM.
> 
> cheers,
> --renato

On 14 June 2016 at 17:40, Chris Bieneman <beanz at apple.com>
wrote:
> The problem comes from that fact that generally builds are structured as
configure -> build. What we actually need is "configure some" ->
"build some" -> "configure some more" -> "build
some more" -> repeat until done.
Well, GCC has been doing this for years, and I have just setup a
simpler version of it here:

http://buildmaster.tcwglab.linaro.org/builders/clang-cmake-aarch64-prototype

To avoid the partial configuration problem, you can build in multiple
stages and increment the level of support in each one until you have a
complete toolchain in the end.

> For our projects the first step of a bootstrap is building a clang that
runs on host supporting the target. After building that clang you need to build
the runtime libraries. For sake of simplicity let's assume we're only
building compiler-rt's current libraries. After building clang the logical
step would be CMake + Ninja for compiler-rt then you're done right? Except,
that doesn't work. You can't *configure* the sanitizer build until after
you've fully configured and built the builtin libraries.
Another reason to move the sanitizers to their own repo.

> If you configure a project before the tools and libraries are available
your configuration system may make the wrong decisions. To put it in more normal
compiler terminology, it is undefined behavior.
So, IIUC, the GCC guys have worked around that problem by defining the
environment with more rigour that we do. It *is* different to build
LLVM with glibc X instead of X+1, and the same is true for all
dependencies.

There isn't a strong reason why we can't do the same, we just don't
do
it because people want to build LLVM in very different architectures.

So, a simpler proposal would be to have a long matrix of tools we need
versus OS we build on and mark the accepted range (or single version)
that you need to have to build it on that particular platform.

Different people might have different toolsets, and that's fine. We
can easily have an entry for Ubuntu and another for Suse, or even one
to Ubuntu X and another for Ubuntu X+1.

Other than doing that, you can't rely on your first stage Clang+LLVM
only build if you have no idea which system compiler was used anyway.
Differences will dilute over the later stages, and ultimately, if you
build enough of them, they'll end up virtually identical. But
validation on those terms become very costly.

> We have viable workarounds for this in our build today. We can extend these
workarounds to make bootstrapping simpler. There are some rough edges that are
caused by limitations in CMake, but there is no fundamental limitation in CMake
that makes this not work.
This is good news! :)

> At Apple we workaround this by having some out-of-tree script goop that
configures and builds clang, then configures and builds the builtins, then
configures and builds the runtimes. Due to complications interfacing with our
internal build system, we'll likely always have that script goop, but
I'd really like to be able to run clean bootstrap builds against open source
without the crazy complications.
That'd probably help other downstream users as well.

> I believe that splitting compiler-rt's sanitizer libraries and builtin
archives into separate projects will allow the project structure to more clearly
represent the way it fits into the project build graph. It is not strictly
necessary for anything, so if people generally don't agree I won't fight
for it.
I think my general feeling of this thread is that most people agree
with the idea. I personally do.

+1 from me.

cheers,
--renato

On 6/14/2016 11:40 AM, Chris Bieneman via llvm-dev
wrote:
> The problem comes from that fact that generally builds are structured as
configure -> build. What we actually need is "configure some" ->
"build some" -> "configure some more" -> "build
some more" -> repeat until done.
> ***Generally speaking, you cannot *configure* a project until all the tools
and libraries your build system is running tests against are built.***
My crazy, totally unrealistic take is that this is a terrible work flow 
that the world has bought in to anyway.  I would much rather tell the 
build system what to do, as opposed to having it guess these values 
based on the environment.  When you get down to first principles, the 
build process is mostly just a series of shell commands.  The build 
scripts / cmake process is generally responsible for constructing, 
choosing, and sequencing these commands.  However, this is mostly just 
unproductive whining, as I don't know of any general purpose build 
systems that make mixed cross-compiles easy.  It's difficult enough to 
convince conventional build systems to add or remove a compiler flag.

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux
Foundation Collaborative Project

On Tue, Jun 14, 2016 at 9:40 AM, Chris Bieneman <beanz at apple.com>
wrote:
> I think I may be explaining the dependency problem poorly. This isn't
so
> much a limitation in CMake, or a problem with our CMake implementation.
> This is a limitation in basically every build configuration management
> system I've ever seen.
>
> The problem comes from that fact that generally builds are structured as
> configure -> build. What we actually need is "configure some"
-> "build
> some" -> "configure some more" -> "build some
more" -> repeat until done.
>
> The problem occurs when you are bootstrapping a toolchain and target. In
> general a bootstrap build involves building a compiler for your target
> (which may or may not match your host), then building the various libraries
> and tools, working up to the full OS.
>
> For our projects the first step of a bootstrap is building a clang that
> runs on host supporting the target. After building that clang you need to
> build the runtime libraries. For sake of simplicity let's assume
we're only
> building compiler-rt's current libraries. After building clang the
logical
> step would be CMake + Ninja for compiler-rt then you're done right?
Except,
> that doesn't work. You can't *configure* the sanitizer build until
after
> you've fully configured and built the builtin libraries.
>
> Pardon the bad formatting here, but this next sentence is really
> important, so I'm going to try and apply some emphasis.
>
> ***Generally speaking, you cannot *configure* a project until all the
> tools and libraries your build system is running tests against are
built.***
>
> If you configure a project before the tools and libraries are available
> your configuration system may make the wrong decisions. To put it in more
> normal compiler terminology, it is undefined behavior.
>
> We have viable workarounds for this in our build today. We can extend
> these workarounds to make bootstrapping simpler. There are some rough edges
> that are caused by limitations in CMake, but there is no fundamental
> limitation in CMake that makes this not work.
>
> At Apple we workaround this by having some out-of-tree script goop that
> configures and builds clang, then configures and builds the builtins, then
> configures and builds the runtimes. Due to complications interfacing with
> our internal build system, we'll likely always have that script goop,
but
> I'd really like to be able to run clean bootstrap builds against open
> source without the crazy complications.
>
> I believe that splitting compiler-rt's sanitizer libraries and builtin
> archives into separate projects will allow the project structure to more
> clearly represent the way it fits into the project build graph. It is not
> strictly necessary for anything, so if people generally don't agree I
won't
> fight for it.
>
If it isn't necessary, I would avoid doing it. There is a cost which is O(#
LLVM developers) and incurred against the entire community and its
constituent companies. This should not be done lightly.

-- Sean Silva

>
> -Chris
>
> > On Jun 12, 2016, at 4:51 AM, Renato Golin <renato.golin at
linaro.org>
> wrote:
> >
> > On 12 June 2016 at 00:35, Sean Silva via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> >>> I think the fundamental distinction needs to be following
dependency
> >>> graphs because if we don’t get rid of the circular dependency
in
> >>> bootstrapping there is no reason to make any changes.
> >>
> >> +1 for what Chandler said here.
> >>
> >> I don't think CMake itself per se cares about actual VCS repo
> breakdown. It
> >> should be possible to bring sanity without any change to VCS
structure.
> >
> > Funny enough "compiler-rt" is a fitting name to anything
that provides
> > run-time libraries that users shouldn't be fiddling with, and that
> > includes builtins, sanitizers, profiling, the lot.
> >
> > Not getting into the name and split bike shed, I agree with Sean that
> > this is a CMake issue, not a directory structure one.
> >
> > Simply put, RT's CMake is a mess.
> >
> > If we can build each component (builtins, sans, profs) independently,
> > then the overall build dependency can follow without a split, by
> > basically creating internal independent targets with a phony target
> > that builds them all being the default.
> >
> > So, if in the main CMake you ask for libc++, RT and unwind, the build
> > dependency will be from specific internal targets in libc++ to other
> > internal targets in RT and so on.
> >
> > This could also be a way out to separate the "builtins"
target into
> > "builtins.<arch>" into different internal targets to
allow
> > cross-compilation. Though, that'd requiring having a complete
> > toolchain for each of the supported target, so it can't be default
on
> > the targets built in LLVM.
> >
> > cheers,
> > --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160614/9c9201d1/attachment.html>