llvm dev - Jun 2016 - [RFC] LLVM Directory Structure Changes (was Re: [PATCH] D20992: [CMake] Add LLVM runtimes directory)

I probably should have stated this in my last email. I see all three of these
initiatives as separate restructuring changes. None of them should need to be
tied to each other or block the others.

> On Jun 10, 2016, at 11:23 AM, Chandler Carruth <chandlerc at
gmail.com> wrote:
> 
> On Fri, Jun 10, 2016 at 10:52 AM Chris Bieneman via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> It seems to me that the feedback here has been generally positive, but a
lot of different ideas have been added to the mix.
> 
> To focus conversation and move things along I'm going to provide a
summary of changes with proposals for rollout.
> 
> Splitting Compiler-RT
> 
> Note that none of the main sanitizer developers have really chimed in
here... It'd be good to actually talk to them first. =]
Agreed. My hope is that some of them may see this thread and chime in.
>  
> If we want to split compiler-rt, which I think makes a lot of sense, I
think the best path forward is to copy the trunk (via svn cp). Copying the
branch is the best way to preserve the history and workflows.
> 
> For naming purposes I would suggest retaining the compiler-rt name for the
builtin libraries, and having a repository named sanitizer-rt for the sanitizer
libraries (this is of course just a suggestion, feel free to bike shed).
> 
> I would very much like a more specific name than 'compiler-rt'. The
genericness of that name is what led to some of the confusion today I suspect.
> 
> I would also suggest not having a hyphen in the name which makes python and
other systems sad (I don't understand why, and I've given up fighting
this battle).
> 
> I think you already used the word that would best describe this:
"builtin libraries".
> 
> However, I'm not sure if splitting (at this point) makes sense. Maybe
it does, but its seems fuzzy to me. The "builtins" will still be a
collection of multiple runtime libraries, all tied to builtin compiler features.
Some will be C/C++ features (the libgcc alternative for EH, type info, and math
stuff). Some will be profiling features and some will be sanitizer features. I
think having a separate repository for the profiling runtimes would probably be
overkill. Maybe sanitizers are big enough to split out, but it seems iffy to me.
I think the big thing that would help would just be better organization *within*
the tree to clearly name the profile, language builtins, and sanitizer
components.
I think the fundamental distinction needs to be following dependency graphs
because if we don’t get rid of the circular dependency in bootstrapping there is
no reason to make any changes. The builtin, profile and safestack library are
all static archives, so they don’t actually require a built libcxx (although
they can rely on the headers). They can all be treated as “builtins”.

The sanitizer runtimes produce dylibs that must be linked against libcxx, so
they cannot be treated as builtins. IMO, following the general rule of thumb
“does this need a linker” is probably close to the right bar.
> 
> Either way, I'd call the thing with profiling and language builtin
runitmes "builtins" before "compiler"-anything.
> 
>  
> Breaking out testing tools
> 
> As I started looking into breaking out the testing tools I realized it is
*much* more complicated than I had first thought. I do think that it is a good
idea to do this, but it is going to be a bigger change than I had originally
thought.
> 
> FWIW, I'm not at all convinced this is a good idea yet. It has some
appeal, but we've tried this before and it created confusion bordering on
chaos. I would definitely decouple these things.
I’ve been discussing this all morning in the halls, and most people in our
office seem to think it would be neat, but is a huge (and separate) project.

-Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160610/f9a754c9/attachment.html>

On Fri, Jun 10, 2016 at 11:41 AM, Chris Bieneman via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I probably should have stated this in my last email. I see all three of
> these initiatives as separate restructuring changes. None of them should
> need to be tied to each other or block the others.
>
>
> On Jun 10, 2016, at 11:23 AM, Chandler Carruth <chandlerc at
gmail.com>
> wrote:
>
> On Fri, Jun 10, 2016 at 10:52 AM Chris Bieneman via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> It seems to me that the feedback here has been generally positive, but
a
>> lot of different ideas have been added to the mix.
>>
>> To focus conversation and move things along I'm going to provide a
>> summary of changes with proposals for rollout.
>>
>> Splitting Compiler-RT
>>
>
> Note that none of the main sanitizer developers have really chimed in
> here... It'd be good to actually talk to them first. =]
>
>
> Agreed. My hope is that some of them may see this thread and chime in.
>
>
>
>> If we want to split compiler-rt, which I think makes a lot of sense, I
>> think the best path forward is to copy the trunk (via svn cp). Copying
the
>> branch is the best way to preserve the history and workflows.
>>
>> For naming purposes I would suggest retaining the compiler-rt name for
>> the builtin libraries, and having a repository named sanitizer-rt for
the
>> sanitizer libraries (this is of course just a suggestion, feel free to
bike
>> shed).
>>
>
> I would very much like a more specific name than 'compiler-rt'. The
> genericness of that name is what led to some of the confusion today I
> suspect.
>
> I would also suggest not having a hyphen in the name which makes python
> and other systems sad (I don't understand why, and I've given up
fighting
> this battle).
>
> I think you already used the word that would best describe this:
"builtin
> libraries".
>
> However, I'm not sure if splitting (at this point) makes sense. Maybe
it
> does, but its seems fuzzy to me. The "builtins" will still be a
collection
> of multiple runtime libraries, all tied to builtin compiler features. Some
> will be C/C++ features (the libgcc alternative for EH, type info, and math
> stuff). Some will be profiling features and some will be sanitizer
> features. I think having a separate repository for the profiling runtimes
> would probably be overkill. Maybe sanitizers are big enough to split out,
> but it seems iffy to me. I think the big thing that would help would just
> be better organization *within* the tree to clearly name the profile,
> language builtins, and sanitizer components.
>
>
> I think the fundamental distinction needs to be following dependency
> graphs because if we don’t get rid of the circular dependency in
> bootstrapping there is no reason to make any changes.
>
+1 for what Chandler said here.

I don't think CMake itself per se cares about actual VCS repo breakdown. It
should be possible to bring sanity without any change to VCS structure.

-- Sean Silva


> The builtin, profile and safestack library are all static archives, so
> they don’t actually require a built libcxx (although they can rely on the
> headers). They can all be treated as “builtins”.
>
> The sanitizer runtimes produce dylibs that must be linked against libcxx,
> so they cannot be treated as builtins. IMO, following the general rule of
> thumb “does this need a linker” is probably close to the right bar.
>
>
> Either way, I'd call the thing with profiling and language builtin
> runitmes "builtins" before "compiler"-anything.
>
>
>
>> Breaking out testing tools
>>
>> As I started looking into breaking out the testing tools I realized it
is
>> *much* more complicated than I had first thought. I do think that it is
a
>> good idea to do this, but it is going to be a bigger change than I had
>> originally thought.
>>
>
> FWIW, I'm not at all convinced this is a good idea yet. It has some
> appeal, but we've tried this before and it created confusion bordering
on
> chaos. I would definitely decouple these things.
>
>
> I’ve been discussing this all morning in the halls, and most people in our
> office seem to think it would be neat, but is a huge (and separate)
project.
>
> -Chris
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160611/be527499/attachment.html>

On 12 June 2016 at 00:35, Sean Silva via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>> I think the fundamental distinction needs to be following dependency
>> graphs because if we don’t get rid of the circular dependency in
>> bootstrapping there is no reason to make any changes.
>
> +1 for what Chandler said here.
>
> I don't think CMake itself per se cares about actual VCS repo
breakdown. It
> should be possible to bring sanity without any change to VCS structure.
Funny enough "compiler-rt" is a fitting name to anything that provides
run-time libraries that users shouldn't be fiddling with, and that
includes builtins, sanitizers, profiling, the lot.

Not getting into the name and split bike shed, I agree with Sean that
this is a CMake issue, not a directory structure one.

Simply put, RT's CMake is a mess.

If we can build each component (builtins, sans, profs) independently,
then the overall build dependency can follow without a split, by
basically creating internal independent targets with a phony target
that builds them all being the default.

So, if in the main CMake you ask for libc++, RT and unwind, the build
dependency will be from specific internal targets in libc++ to other
internal targets in RT and so on.

This could also be a way out to separate the "builtins" target into
"builtins.<arch>" into different internal targets to allow
cross-compilation. Though, that'd requiring having a complete
toolchain for each of the supported target, so it can't be default on
the targets built in LLVM.

cheers,
--renato