llvm dev - Apr 2020 - Splitting up Type.h: Good idea, bad idea?

Hello Clang folks,

I was using -ftime-trace to see where the compiler spends time parsing
clang's own headers, and it pointed me to Type.h. Many AST headers need
QualType to be complete, but they do not need the full type class
hierarchy. To improve compile time, I have attempted to create a new
header, QualType.h, that defines only the QualType wrapper class. I have
started to transition popular clang headers (Decl.h, Expr.h, ASTContext.h,
etc) to just use QualType.h. I got pretty far with this, and pushed the
results to github:
https://github.com/llvm/llvm-project/compare/master...rnk:split-qual-type-prune

You can see it is pretty involved / invasive. Many inline methods had to be
sunk to cpp files, and this may affect performance. So far, I have nothing
to show for it in build time gains, because there are still too many Type.h
includes.

At this point I am wondering if this is worth doing at all. I figured I
would take a break and ask for some opinions. If people generally feel this
is worth pursuing, I'll break that branch up and upload it to phab in
incremental pieces.

The next most popular header seems to be CanonicalType.h, and it appears to
be pretty strongly coupled to Type.h. It essentially reiterates all of the
derived classes of Type, so I'm not sure how to break this link yet.

Reid
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200407/5edf19bb/attachment.html>

On 7 Apr 2020, at 13:27, Reid Kleckner wrote:
> Hello Clang folks,
Why is this on llvm-dev instead of cfe-dev?
> I was using -ftime-trace to see where the compiler spends time parsing
> clang's own headers, and it pointed me to Type.h. Many AST headers 
> need
> QualType to be complete, but they do not need the full type class
> hierarchy. To improve compile time, I have attempted to create a new
> header, QualType.h, that defines only the QualType wrapper class. I 
> have
> started to transition popular clang headers (Decl.h, Expr.h, 
> ASTContext.h,
> etc) to just use QualType.h. I got pretty far with this, and pushed 
> the
> results to github:
>
https://github.com/llvm/llvm-project/compare/master...rnk:split-qual-type-prune
>
> You can see it is pretty involved / invasive. Many inline methods had 
> to be
> sunk to cpp files, and this may affect performance. So far, I have 
> nothing
> to show for it in build time gains, because there are still too many 
> Type.h
> includes.
How many translation units actually don’t ultimately need the type
definitions?  Because this achieves nothing if almost every translation
unit ends up including Type.h anyway.

John.

+cfe-dev.

Splitting this header up makes sense to me.  Types.h was not intended to be the
7000 line monster it is now :-).

Splitting QualType out to its own header makes a lot of sense to me, but would
it make sense to further split it up somehow?  For example, one could split the
C-like types, from the C++-like types, from the extensions.  I’m not sure if
that would be useful though.

-Chris
> On Apr 7, 2020, at 10:27 AM, Reid Kleckner via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hello Clang folks,
> 
> I was using -ftime-trace to see where the compiler spends time parsing
clang's own headers, and it pointed me to Type.h. Many AST headers need
QualType to be complete, but they do not need the full type class hierarchy. To
improve compile time, I have attempted to create a new header, QualType.h, that
defines only the QualType wrapper class. I have started to transition popular
clang headers (Decl.h, Expr.h, ASTContext.h, etc) to just use QualType.h. I got
pretty far with this, and pushed the results to github:
>
https://github.com/llvm/llvm-project/compare/master...rnk:split-qual-type-prune
<https://github.com/llvm/llvm-project/compare/master...rnk:split-qual-type-prune>
> 
> You can see it is pretty involved / invasive. Many inline methods had to be
sunk to cpp files, and this may affect performance. So far, I have nothing to
show for it in build time gains, because there are still too many Type.h
includes.
> 
> At this point I am wondering if this is worth doing at all. I figured I
would take a break and ask for some opinions. If people generally feel this is
worth pursuing, I'll break that branch up and upload it to phab in
incremental pieces.
> 
> The next most popular header seems to be CanonicalType.h, and it appears to
be pretty strongly coupled to Type.h. It essentially reiterates all of the
derived classes of Type, so I'm not sure how to break this link yet.
> 
> Reid
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200407/8da4c0e8/attachment.html>

On Tue, Apr 7, 2020 at 11:35 AM Chris Lattner <clattner at nondot.org>
wrote:
> +cfe-dev.
>
Sorry about it going to llvm-dev. I typed cfe-dev<enter> into the To: box
and got llvm-dev.

> Splitting this header up makes sense to me.  Types.h was not intended to
> be the 7000 line monster it is now :-).
>
> Splitting QualType out to its own header makes a lot of sense to me, but
> would it make sense to further split it up somehow?  For example, one could
> split the C-like types, from the C++-like types, from the extensions.  I’m
> not sure if that would be useful though.
>
I think that's a good idea. I think there are some key types that everyone
uses (ReferenceType, PointerType, FunctionType). The Decl and Stmt class
hierarchy currently often use these types from inline methods.

On Tue, Apr 7, 2020 at 10:34 AM John McCall <rjmccall at apple.com> wrote:
> How many translation units actually don’t ultimately need the type
> definitions?  Because this achieves nothing if almost every translation
> unit ends up including Type.h anyway.

It's hard to know this without doing it. My plan was to do it, hope to get
early good results, and use that to motivate finishing the project. But, my
lack of results is making me reconsider the whole project.

The IWYU tool exists, but the results are generally considered unusable. We
could try to run it on the codebase and see how often it thinks Type.h
should be included. Part of the project is refactoring the code to not use
Type.h.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200407/77ab8f81/attachment.html>