llvm dev - Jul 2019 - Tablegen ridiculously slow when compiling for Debug

This is also the case with the Visual Studio generators. Custom commands in
a single cmake file essentially get written out line by line into a single
batch file that gets processed as a custom build step. In the VS case this
means that it can, for example, run X86 and Aarch64 tablegen steps in
parallel with each other but all of the individual X86 invocations get
processed serially. I can well imagine that the Xcode situation is similar
although I've no experience with it myself to know for sure.

As previously mentioned, the best solution is probably to try to adjust
your workflow to use the Ninja (
https://ninja-build.org
 ) CMake generator if at all possible.  It's a bit of an adjustment but it
does work very nicely with the LLVM build system.

-Greg


On Sun, 30 Jun 2019 at 12:08, Nicolai Hähnle via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Are you saying that the TableGen execution isn't parallelized? That
> seems like an obvious Xcode-specific problem...
>
> The TableGen executions are parallelized with cmake/ninja just fine.
>
> Cheers,
> Nicolai
>
> On 30.06.19 11:28, Joan Lluch via llvm-dev wrote:
> > Hi Praveen,
> >
> > Thanks for the tip, but Xcode seems to spend all the time running
> > tablegen "custom shell scripts", one by one at a time, not
linking.
> > Linking is actually very fast, possibly less than a second. The
> > “scripts” that take longer are  “AArch64CommonTableGen" and
> > “AMDGPUCommonTableGen”.  As said this is on LLVM 9.0.
> >
> > However, on LLVM 7.0.1, the same process takes just 5-6 seconds in
> > total, with individual “scripts” taking significantly less than 1
second
> > each. There must be some difference between LLVM 9.0 and LLVM 7.0 that
> > might cause this (?)
> >
> > John
> >
> >
> >> On 30 Jun 2019, at 11:17, Praveen Velliengiri
> >> <praveenvelliengiri at gmail.com <mailto:praveenvelliengiri
at gmail.com>>
> >> wrote:
> >>
> >> *
> >> *
> >> cmake *BUILD_SHARED_LIBS* option, it builds llvm as .so not as .a.
It
> >> will use less memory during linking so you can increase the link
> >> threads and your build time will be lesser.
> >> Check this in : https://llvm.org/docs/CMake.html
> >>
> >> **
> >> **
> >>
> >> On Sun, 30 Jun 2019 at 14:42, Joan Lluch <joan.lluch at
icloud.com
> >> <mailto:joan.lluch at icloud.com>> wrote:
> >>
> >>     Hi Praveen,
> >>
> >>     Please, can you elaborate on this?. What do do mean by
“building
> >>     as shared objects”.
> >>
> >>     Thanks,
> >>
> >>     John
> >>
> >>
> >>
> >>>     On 30 Jun 2019, at 07:32, Praveen Velliengiri
> >>>     <praveenvelliengiri at gmail.com
> >>>     <mailto:praveenvelliengiri at gmail.com>> wrote:
> >>>
> >>>     Maybe try building llvm as a shared objects..
> >>>
> >>>     On Jun 30, 2019 1:30 AM, "Joan Lluch via
llvm-dev"
> >>>     <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
> >>>
> >>>         Hi Florian,
> >>>
> >>>         Ok, I ran this:
> >>>
> >>>         cmake -S LLVM -DCMAKE_INSTALL_PREFIX=INSTALL
> >>>         -DLLVM_OPTIMIZED_TABLEGEN=On -G Xcode
> >>>
> >>>         Compiled it again from clean, and the situation is
worse than
> >>>         before. Incremental builds take an incredible amount
of time
> >>>         stuck in running Tablegen scripts for all targets. Now
this
> >>>         happens both in Release and Debug configurations. Just
before
> >>>         this, at least Release compiled fine, but that’s no
longer
> >>>         the case.
> >>>
> >>>         Any other suggestions?  What could actually cause
this?
> >>>
> >>>         Thanks
> >>>         John
> >>>
> >>>
> >>>
> >>>>         On 29 Jun 2019, at 19:37, Florian Hahn
> >>>>         <florian_hahn at apple.com
<mailto:florian_hahn at apple.com>>
> wrote:
> >>>>
> >>>>         Hi,
> >>>>
> >>>>>         On Jun 29, 2019, at 18:26, Joan Lluch via
llvm-dev
> >>>>>         <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
> >>>>>         wrote:
> >>>>>
> >>>>>         Hi all,
> >>>>>
> >>>>>         On LLVM version 7.0.1, incremental builds are
very fast for
> >>>>>         both Release and Debug. I’m compiling with
Xcode
> >>>>>
> >>>>>         I recently downloaded LLVM 9.0 from the
LLVM-mirror Github
> >>>>>         repository and found that Incremental
"Debug” builds take a
> >>>>>         ridiculously long time due to Tablegen taking
ages
> >>>>>         (literally more than 10 minutes) to generate
files. This
> >>>>>         makes it totally unusable for debug purposes.
However,
> >>>>>         incremental ‘Release’ builds only take a few
seconds.
> >>>>>
> >>>>>         Why is that?. Any suggestions?.
> >>>>
> >>>>
> >>>>
> >>>>         You could give setting LLVM_OPTIMIZED_TABLEGEN a
try
> >>>>         (https://llvm.org/docs/CMake.html).
> >>>>
> >>>>         Cheers,
> >>>>         Florian
> >>>
> >>>
> >>>         _______________________________________________
> >>>         LLVM Developers mailing list
> >>>         llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
> >>>        
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>>
> >>
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> --
> Lerne, wie die Welt wirklich ist,
> Aber vergiss niemals, wie sie sein sollte.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190630/f7da32d3/attachment.html>

Hi Greg,

I tried to setup Ninja before on my mac but I mush have done something wrong and
I didn’t manage to get it work. I’m not familiarised at all with the procedures
involved. I may try that again to see If I have some luck though. It’s a pity
that LLVM is not particularly friendly with familiar IDEs such as xCode on macs
and Visual Studio on windows.

John
> On 30 Jun 2019, at 17:43, Greg Bedwell <gregbedwell at gmail.com>
wrote:
> 
> This is also the case with the Visual Studio generators. Custom commands in
a single cmake file essentially get written out line by line into a single batch
file that gets processed as a custom build step. In the VS case this means that
it can, for example, run X86 and Aarch64 tablegen steps in parallel with each
other but all of the individual X86 invocations get processed serially. I can
well imagine that the Xcode situation is similar although I've no experience
with it myself to know for sure.
> 
> As previously mentioned, the best solution is probably to try to adjust
your workflow to use the Ninja (
> https://ninja-build.org <https://ninja-build.org/> ) CMake generator
if at all possible.  It's a bit of an adjustment but it does work very
nicely with the LLVM build system.
> 
> -Greg
> 
> 
> On Sun, 30 Jun 2019 at 12:08, Nicolai Hähnle via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Are you saying that the TableGen execution isn't parallelized? That 
> seems like an obvious Xcode-specific problem...
> 
> The TableGen executions are parallelized with cmake/ninja just fine.
> 
> Cheers,
> Nicolai
> 
> On 30.06.19 11:28, Joan Lluch via llvm-dev wrote:
> > Hi Praveen,
> > 
> > Thanks for the tip, but Xcode seems to spend all the time running 
> > tablegen "custom shell scripts", one by one at a time, not
linking.
> > Linking is actually very fast, possibly less than a second. The 
> > “scripts” that take longer are  “AArch64CommonTableGen" and 
> > “AMDGPUCommonTableGen”.  As said this is on LLVM 9.0.
> > 
> > However, on LLVM 7.0.1, the same process takes just 5-6 seconds in 
> > total, with individual “scripts” taking significantly less than 1
second
> > each. There must be some difference between LLVM 9.0 and LLVM 7.0 that
> > might cause this (?)
> > 
> > John
> > 
> > 
> >> On 30 Jun 2019, at 11:17, Praveen Velliengiri 
> >> <praveenvelliengiri at gmail.com <mailto:praveenvelliengiri
at gmail.com> <mailto:praveenvelliengiri at gmail.com
<mailto:praveenvelliengiri at gmail.com>>>
> >> wrote:
> >>
> >> *
> >> *
> >> cmake *BUILD_SHARED_LIBS* option, it builds llvm as .so not as .a.
It
> >> will use less memory during linking so you can increase the link 
> >> threads and your build time will be lesser.
> >> Check this in : https://llvm.org/docs/CMake.html
<https://llvm.org/docs/CMake.html>
> >>
> >> **
> >> **
> >>
> >> On Sun, 30 Jun 2019 at 14:42, Joan Lluch <joan.lluch at
icloud.com <mailto:joan.lluch at icloud.com>
> >> <mailto:joan.lluch at icloud.com <mailto:joan.lluch at
icloud.com>>> wrote:
> >>
> >>     Hi Praveen,
> >>
> >>     Please, can you elaborate on this?. What do do mean by
“building
> >>     as shared objects”.
> >>
> >>     Thanks,
> >>
> >>     John
> >>
> >>
> >>
> >>>     On 30 Jun 2019, at 07:32, Praveen Velliengiri
> >>>     <praveenvelliengiri at gmail.com
<mailto:praveenvelliengiri at gmail.com>
> >>>     <mailto:praveenvelliengiri at gmail.com
<mailto:praveenvelliengiri at gmail.com>>> wrote:
> >>>
> >>>     Maybe try building llvm as a shared objects..
> >>>
> >>>     On Jun 30, 2019 1:30 AM, "Joan Lluch via
llvm-dev"
> >>>     <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org> <mailto:llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>>> wrote:
> >>>
> >>>         Hi Florian,
> >>>
> >>>         Ok, I ran this:
> >>>
> >>>         cmake -S LLVM -DCMAKE_INSTALL_PREFIX=INSTALL
> >>>         -DLLVM_OPTIMIZED_TABLEGEN=On -G Xcode
> >>>
> >>>         Compiled it again from clean, and the situation is
worse than
> >>>         before. Incremental builds take an incredible amount
of time
> >>>         stuck in running Tablegen scripts for all targets. Now
this
> >>>         happens both in Release and Debug configurations. Just
before
> >>>         this, at least Release compiled fine, but that’s no
longer
> >>>         the case.
> >>>
> >>>         Any other suggestions?  What could actually cause
this?
> >>>
> >>>         Thanks
> >>>         John
> >>>
> >>>
> >>>
> >>>>         On 29 Jun 2019, at 19:37, Florian Hahn
> >>>>         <florian_hahn at apple.com
<mailto:florian_hahn at apple.com> <mailto:florian_hahn at apple.com
<mailto:florian_hahn at apple.com>>> wrote:
> >>>>
> >>>>         Hi,
> >>>>
> >>>>>         On Jun 29, 2019, at 18:26, Joan Lluch via
llvm-dev
> >>>>>         <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org> <mailto:llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>>
> >>>>>         wrote:
> >>>>>
> >>>>>         Hi all,
> >>>>>
> >>>>>         On LLVM version 7.0.1, incremental builds are
very fast for
> >>>>>         both Release and Debug. I’m compiling with
Xcode
> >>>>>
> >>>>>         I recently downloaded LLVM 9.0 from the
LLVM-mirror Github
> >>>>>         repository and found that Incremental
"Debug” builds take a
> >>>>>         ridiculously long time due to Tablegen taking
ages
> >>>>>         (literally more than 10 minutes) to generate
files. This
> >>>>>         makes it totally unusable for debug purposes.
However,
> >>>>>         incremental ‘Release’ builds only take a few
seconds.
> >>>>>
> >>>>>         Why is that?. Any suggestions?.
> >>>>
> >>>>
> >>>>
> >>>>         You could give setting LLVM_OPTIMIZED_TABLEGEN a
try
> >>>>         (https://llvm.org/docs/CMake.html
<https://llvm.org/docs/CMake.html>).
> >>>>
> >>>>         Cheers,
> >>>>         Florian
> >>>
> >>>
> >>>         _______________________________________________
> >>>         LLVM Developers mailing list
> >>>         llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org> <mailto:llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>>
> >>>        
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> >>>
> >>
> > 
> > 
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> > 
> 
> -- 
> Lerne, wie die Welt wirklich ist,
> Aber vergiss niemals, wie sie sein sollte.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190630/e6840f34/attachment.html>

Hey Joan,

When looking for build support it is really useful to include a bunch of
information about your build up front. Knowing that you are on macOS, and using
the Xcode generator are really useful.

On macOS, BUILD_SHARED_LIBS won’t really help much because the default linker
(ld64) is pretty good. Using an IDE generator and setting
LLVM_USE_OPTIMIZED_TABLEGEN will kill your release builds.

In general Xcode takes 2x-3x longer than Ninja for incremental builds, and
1.5x-2x as long for clean builds. Lots of people use the Xcode generator to
create a project file for navigation and editing, but most people I know doing
LLVM development on macOS use Ninja for their builds.

-Chris
> On Jun 30, 2019, at 12:18 PM, Joan Lluch via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi Greg,
> 
> I tried to setup Ninja before on my mac but I mush have done something
wrong and I didn’t manage to get it work. I’m not familiarised at all with the
procedures involved. I may try that again to see If I have some luck though.
It’s a pity that LLVM is not particularly friendly with familiar IDEs such as
xCode on macs and Visual Studio on windows.
> 
> John
> 
>> On 30 Jun 2019, at 17:43, Greg Bedwell <gregbedwell at gmail.com>
wrote:
>> 
>> This is also the case with the Visual Studio generators. Custom
commands in a single cmake file essentially get written out line by line into a
single batch file that gets processed as a custom build step. In the VS case
this means that it can, for example, run X86 and Aarch64 tablegen steps in
parallel with each other but all of the individual X86 invocations get processed
serially. I can well imagine that the Xcode situation is similar although
I've no experience with it myself to know for sure.
>> 
>> As previously mentioned, the best solution is probably to try to adjust
your workflow to use the Ninja (
>> https://ninja-build.org
>>  ) CMake generator if at all possible.  It's a bit of an adjustment
but it does work very nicely with the LLVM build system.
>> 
>> -Greg
>> 
>> 
>>> On Sun, 30 Jun 2019 at 12:08, Nicolai Hähnle via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>> Are you saying that the TableGen execution isn't parallelized?
That
>>> seems like an obvious Xcode-specific problem...
>>> 
>>> The TableGen executions are parallelized with cmake/ninja just
fine.
>>> 
>>> Cheers,
>>> Nicolai
>>> 
>>> On 30.06.19 11:28, Joan Lluch via llvm-dev wrote:
>>> > Hi Praveen,
>>> > 
>>> > Thanks for the tip, but Xcode seems to spend all the time
running
>>> > tablegen "custom shell scripts", one by one at a
time, not linking.
>>> > Linking is actually very fast, possibly less than a second.
The
>>> > “scripts” that take longer are  “AArch64CommonTableGen"
and
>>> > “AMDGPUCommonTableGen”.  As said this is on LLVM 9.0.
>>> > 
>>> > However, on LLVM 7.0.1, the same process takes just 5-6
seconds in
>>> > total, with individual “scripts” taking significantly less
than 1 second
>>> > each. There must be some difference between LLVM 9.0 and LLVM
7.0 that
>>> > might cause this (?)
>>> > 
>>> > John
>>> > 
>>> > 
>>> >> On 30 Jun 2019, at 11:17, Praveen Velliengiri 
>>> >> <praveenvelliengiri at gmail.com
<mailto:praveenvelliengiri at gmail.com>>
>>> >> wrote:
>>> >>
>>> >> *
>>> >> *
>>> >> cmake *BUILD_SHARED_LIBS* option, it builds llvm as .so
not as .a. It
>>> >> will use less memory during linking so you can increase
the link
>>> >> threads and your build time will be lesser.
>>> >> Check this in : https://llvm.org/docs/CMake.html
>>> >>
>>> >> **
>>> >> **
>>> >>
>>> >> On Sun, 30 Jun 2019 at 14:42, Joan Lluch <joan.lluch at
icloud.com
>>> >> <mailto:joan.lluch at icloud.com>> wrote:
>>> >>
>>> >>     Hi Praveen,
>>> >>
>>> >>     Please, can you elaborate on this?. What do do mean by
“building
>>> >>     as shared objects”.
>>> >>
>>> >>     Thanks,
>>> >>
>>> >>     John
>>> >>
>>> >>
>>> >>
>>> >>>     On 30 Jun 2019, at 07:32, Praveen Velliengiri
>>> >>>     <praveenvelliengiri at gmail.com
>>> >>>     <mailto:praveenvelliengiri at gmail.com>>
wrote:
>>> >>>
>>> >>>     Maybe try building llvm as a shared objects..
>>> >>>
>>> >>>     On Jun 30, 2019 1:30 AM, "Joan Lluch via
llvm-dev"
>>> >>>     <llvm-dev at lists.llvm.org <mailto:llvm-dev
at lists.llvm.org>> wrote:
>>> >>>
>>> >>>         Hi Florian,
>>> >>>
>>> >>>         Ok, I ran this:
>>> >>>
>>> >>>         cmake -S LLVM -DCMAKE_INSTALL_PREFIX=INSTALL
>>> >>>         -DLLVM_OPTIMIZED_TABLEGEN=On -G Xcode
>>> >>>
>>> >>>         Compiled it again from clean, and the
situation is worse than
>>> >>>         before. Incremental builds take an incredible
amount of time
>>> >>>         stuck in running Tablegen scripts for all
targets. Now this
>>> >>>         happens both in Release and Debug
configurations. Just before
>>> >>>         this, at least Release compiled fine, but
that’s no longer
>>> >>>         the case.
>>> >>>
>>> >>>         Any other suggestions?  What could actually
cause this?
>>> >>>
>>> >>>         Thanks
>>> >>>         John
>>> >>>
>>> >>>
>>> >>>
>>> >>>>         On 29 Jun 2019, at 19:37, Florian Hahn
>>> >>>>         <florian_hahn at apple.com
<mailto:florian_hahn at apple.com>> wrote:
>>> >>>>
>>> >>>>         Hi,
>>> >>>>
>>> >>>>>         On Jun 29, 2019, at 18:26, Joan Lluch
via llvm-dev
>>> >>>>>         <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
>>> >>>>>         wrote:
>>> >>>>>
>>> >>>>>         Hi all,
>>> >>>>>
>>> >>>>>         On LLVM version 7.0.1, incremental
builds are very fast for
>>> >>>>>         both Release and Debug. I’m compiling
with Xcode
>>> >>>>>
>>> >>>>>         I recently downloaded LLVM 9.0 from
the LLVM-mirror Github
>>> >>>>>         repository and found that Incremental
"Debug” builds take a
>>> >>>>>         ridiculously long time due to Tablegen
taking ages
>>> >>>>>         (literally more than 10 minutes) to
generate files. This
>>> >>>>>         makes it totally unusable for debug
purposes. However,
>>> >>>>>         incremental ‘Release’ builds only take
a few seconds.
>>> >>>>>
>>> >>>>>         Why is that?. Any suggestions?.
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>         You could give setting
LLVM_OPTIMIZED_TABLEGEN a try
>>> >>>>         (https://llvm.org/docs/CMake.html).
>>> >>>>
>>> >>>>         Cheers,
>>> >>>>         Florian
>>> >>>
>>> >>>
>>> >>>        
_______________________________________________
>>> >>>         LLVM Developers mailing list
>>> >>>         llvm-dev at lists.llvm.org <mailto:llvm-dev
at lists.llvm.org>
>>> >>>        
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> >>>
>>> >>
>>> > 
>>> > 
>>> > _______________________________________________
>>> > LLVM Developers mailing list
>>> > llvm-dev at lists.llvm.org
>>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> > 
>>> 
>>> -- 
>>> Lerne, wie die Welt wirklich ist,
>>> Aber vergiss niemals, wie sie sein sollte.
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190701/bb5708d7/attachment.html>