llvm dev - Jun 2016 - [RFC] LLVM Directory Structure Changes (was Re: [PATCH] D20992: [CMake] Add LLVM runtimes directory)

On 9 June 2016 at 19:13, Chris Bieneman <beanz at apple.com>
wrote:
>> On 9 June 2016 at 18:49, Justin Bogner via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>>> This all seems pretty sensible. Should we also use the opportunity
to
>>> split compiler-rt's builtins and profiling/sanitizer/etc
runtimes, since
>>> we'll be moving things around anyway?
>>
>> Also be good to make Compiler-RT and libc++ cross-compile for multiple
>> targets... :/
>
> Yes, that is part of the eventual goal for this. I *really* want to be able
to build a full cross-compiler and runtime stack from a single CMake invocation.
Ideally supporting /n/ targets.
>
> I have lots of ideas in this area, but it is being kept in mind.
Thanks Chris,

A big odd user of RT is Android, and they're having a lot of trouble
cross-compiling, so they ended up creating their own build system.

I'm copying Steve that has had more than his share of problems, maybe
a sync on how we both want to compile RT wouldn't hurt.

My goal is to get Android compiling RT together with Clang in the
exact same way we do, so we only need to maintain one toolchain.

cheers,
--renato

On Thu, Jun 9, 2016 at 12:06 PM, Renato Golin <renato.golin at linaro.org>
wrote:
> On 9 June 2016 at 19:13, Chris Bieneman <beanz at apple.com> wrote:
> >> On 9 June 2016 at 18:49, Justin Bogner via llvm-dev
> >> <llvm-dev at lists.llvm.org> wrote:
> >>> This all seems pretty sensible. Should we also use the
opportunity to
> >>> split compiler-rt's builtins and profiling/sanitizer/etc
runtimes,
> since
> >>> we'll be moving things around anyway?
> >>
> >> Also be good to make Compiler-RT and libc++ cross-compile for
multiple
> >> targets... :/
> >
> > Yes, that is part of the eventual goal for this. I *really* want to be
> able to build a full cross-compiler and runtime stack from a single CMake
> invocation. Ideally supporting /n/ targets.
> >
> > I have lots of ideas in this area, but it is being kept in mind.
>
> Thanks Chris,
>
> A big odd user of RT is Android, and they're having a lot of trouble
> cross-compiling, so they ended up creating their own build system.
>
Eh, the build system for Android existed before people were using cmake in
LLVM (i.e. it was based on the old autotools version for LLVM builds). At
the time, nearly all the LLVM components that Android was using were
trivially cross-compiled (well, except for TableGen, but even that isn't
hard to write make rules for). Now that LLVM is a more important chunk of
Android (and growing increasingly more complex), I agree that it isn't the
best to have to maintain a parallel build system that isn't even in
upstream.

>
> I'm copying Steve that has had more than his share of problems, maybe
> a sync on how we both want to compile RT wouldn't hurt.
>
I actually am not sure how we want to compile RT. I am going to have to
hack around the existing state of things for quite some time before
converging on what upstream is doing because Android can't really afford to
not make progress (and we have more than our share of other
problems/deadlines to make this lower priority). I also still don't know
enough about cmake to really understand why cross-compiling is so
difficult, but I can see where the builtins combined with the sanitizers
cause problems (due to different true dependencies on libcxx, etc.). I
think splitting them apart is a great first step, as I already have to
essentially do that for my new builds today (i.e. configure/build runtime,
then configure/build sanitizers later on). Other than that, I am not sure I
have any idea what to do after that.

>
> My goal is to get Android compiling RT together with Clang in the
> exact same way we do, so we only need to maintain one toolchain.
>
Do you have a pointer to how you build these things today? I have searched
for other users of compiler-rt for non-x86 platforms and nearly every
configuration I saw was relying on terrible hacks (i.e. Chromium doesn't
bother to use their new cross-compiler, they just rely on a pre-existing
Android NDK to cross-compile compiler-rt because it is too hard to
configure otherwise). I concluded that because everyone else is mostly
cross-compiling Android targets incorrectly, that I don't particularly feel
compelled to conform to a truly clean upstream build for compiler-rt right
now.

Thanks,
Steve

>
> cheers,
> --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160609/5e8c915d/attachment.html>

On 9 June 2016 at 20:26, Stephen Hines <srhines at google.com>
wrote:
> Eh, the build system for Android existed before people were using cmake in
> LLVM (i.e. it was based on the old autotools version for LLVM builds).
Sorry, I meant "ended up using their own build system". Though,
that's
also not the same thing, since you guys use your build system for
pretty much anything, right?

It may work for most other Android dependencies, but LLVM has a lot of
internal cross dependency and idiosyncratic build system, making
back-porting very hard if we disagree on how to build things.

> I concluded that because everyone else is mostly cross-compiling
> Android targets incorrectly, that I don't particularly feel compelled
to
> conform to a truly clean upstream build for compiler-rt right now.
AFAIK, no one has a good solution, and we definitely don't have a
universal one. But I believe whatever Chris is planning to has a high
potential of being that "one true way" (tm).

I don't like that the sanitizers are in RT in the same way I didn't
like the unwinder was there. The sanitizers cross-depend on Clang,
which the builtins don't have to. Unwind depends on libc++abi, while
neither of the other two do.

But the builtins are weird on their own, and how they pick optimised
versions inside targets directories is the kind of idea that probably
sounded brilliant before implementation, and everyone sighed
afterwards. :)

Even if Android doesn't build LLVM in the upstream way now, knowing
how the plans are progressing will allow you guys to plan for the
future and share concerns, so that both sides know all issues before
agreeing on a long term plan.

cheers,
--renato