llvm dev - Mar 2016 - llvm and clang are getting slower

I have noticed that LLVM doesn't seem to "like" large functions,
as a
general rule. Admittedly, my experience is similar with gcc, so I'm not
sure it's something that can be easily fixed. And I'm probably sounding
like a broken record, because I have said this before.

My experience is that the time it takes to compile something is growing
above linear with size of function.

Of course, the LLVM code is growing over time, both to support more
features and to support more architectures, new processor types and
instruction sets, at least of which will lead to larger functions in
general [and this is the function "after inlining", so splitting small
'called once' functions out doesn't really help much].

I will have a little play to see if I can identify more of a cuplrit [at
the very least if it's "large basic blocks" or "large
functions" that is
the problem] - of course, this could be unrelated and irellevant to the
problem Daniel is pointing at, and it may or may not be easily resolved...

--
Mats

On 8 March 2016 at 18:42, Richard Smith via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Tue, Mar 8, 2016 at 8:13 AM, Rafael Espíndola
> <llvm-dev at lists.llvm.org> wrote:
> > I have just benchmarked building trunk llvm and clang in Debug,
> > Release and LTO modes (see the attached scrip for the cmake lines).
> >
> > The compilers used were clang 3.5, 3.6, 3.7, 3.8 and trunk. In all
> > cases I used the system libgcc and libstdc++.
> >
> > For release builds there is a monotonic increase in each version. From
> > 163 minutes with 3.5 to 212 minutes with trunk. For comparison, gcc
> > 5.3.2 takes 205 minutes.
> >
> > Debug and LTO show an improvement in 3.7, but have regressed again in
> 3.8.
>
> I'm curious how these times divide across Clang and various parts of
> LLVM; rerunning with -ftime-report and summing the numbers across all
> compiles could be interesting.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160308/109754cf/attachment.html>

On Tue, Mar 8, 2016 at 10:55 AM, mats petersson via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I have noticed that LLVM doesn't seem to "like" large
functions, as a
> general rule. Admittedly, my experience is similar with gcc, so I'm not
> sure it's something that can be easily fixed. And I'm probably
sounding
> like a broken record, because I have said this before.
>
> My experience is that the time it takes to compile something is growing
> above linear with size of function.
>
>
The number of BBs -- Kosyia can point you to the compile time bug that is
exposed by asan .

David


> Of course, the LLVM code is growing over time, both to support more
> features and to support more architectures, new processor types and
> instruction sets, at least of which will lead to larger functions in
> general [and this is the function "after inlining", so splitting
small
> 'called once' functions out doesn't really help much].
>
> I will have a little play to see if I can identify more of a cuplrit [at
> the very least if it's "large basic blocks" or "large
functions" that is
> the problem] - of course, this could be unrelated and irellevant to the
> problem Daniel is pointing at, and it may or may not be easily resolved...
>
> --
> Mats
>
> On 8 March 2016 at 18:42, Richard Smith via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On Tue, Mar 8, 2016 at 8:13 AM, Rafael Espíndola
>> <llvm-dev at lists.llvm.org> wrote:
>> > I have just benchmarked building trunk llvm and clang in Debug,
>> > Release and LTO modes (see the attached scrip for the cmake
lines).
>> >
>> > The compilers used were clang 3.5, 3.6, 3.7, 3.8 and trunk. In all
>> > cases I used the system libgcc and libstdc++.
>> >
>> > For release builds there is a monotonic increase in each version.
From
>> > 163 minutes with 3.5 to 212 minutes with trunk. For comparison,
gcc
>> > 5.3.2 takes 205 minutes.
>> >
>> > Debug and LTO show an improvement in 3.7, but have regressed again
in
>> 3.8.
>>
>> I'm curious how these times divide across Clang and various parts
of
>> LLVM; rerunning with -ftime-report and summing the numbers across all
>> compiles could be interesting.
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160308/f28407cc/attachment.html>

On Tue, Mar 8, 2016 at 11:18 AM, Xinliang David Li via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>
> On Tue, Mar 8, 2016 at 10:55 AM, mats petersson via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> I have noticed that LLVM doesn't seem to "like" large
functions, as a
>> general rule. Admittedly, my experience is similar with gcc, so I'm
not
>> sure it's something that can be easily fixed. And I'm probably
sounding
>> like a broken record, because I have said this before.
>>
>> My experience is that the time it takes to compile something is growing
>> above linear with size of function.
>>
>>
> The number of BBs -- Kosyia can point you to the compile time bug that is
> exposed by asan .
>
I believe we also have some superlinear behavior with BB size as well.

-- Sean Silva

>
> David
>
>
>
>> Of course, the LLVM code is growing over time, both to support more
>> features and to support more architectures, new processor types and
>> instruction sets, at least of which will lead to larger functions in
>> general [and this is the function "after inlining", so
splitting small
>> 'called once' functions out doesn't really help much].
>>
>> I will have a little play to see if I can identify more of a cuplrit
[at
>> the very least if it's "large basic blocks" or
"large functions" that is
>> the problem] - of course, this could be unrelated and irellevant to the
>> problem Daniel is pointing at, and it may or may not be easily
resolved...
>>
>> --
>> Mats
>>
>> On 8 March 2016 at 18:42, Richard Smith via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> On Tue, Mar 8, 2016 at 8:13 AM, Rafael Espíndola
>>> <llvm-dev at lists.llvm.org> wrote:
>>> > I have just benchmarked building trunk llvm and clang in
Debug,
>>> > Release and LTO modes (see the attached scrip for the cmake
lines).
>>> >
>>> > The compilers used were clang 3.5, 3.6, 3.7, 3.8 and trunk. In
all
>>> > cases I used the system libgcc and libstdc++.
>>> >
>>> > For release builds there is a monotonic increase in each
version. From
>>> > 163 minutes with 3.5 to 212 minutes with trunk. For
comparison, gcc
>>> > 5.3.2 takes 205 minutes.
>>> >
>>> > Debug and LTO show an improvement in 3.7, but have regressed
again in
>>> 3.8.
>>>
>>> I'm curious how these times divide across Clang and various
parts of
>>> LLVM; rerunning with -ftime-report and summing the numbers across
all
>>> compiles could be interesting.
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160308/f75b2e5b/attachment.html>

On Tue, Mar 8, 2016 at 11:18 AM, Xinliang David Li <xinliangli at
gmail.com>
wrote:
>
>
> On Tue, Mar 8, 2016 at 10:55 AM, mats petersson via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> I have noticed that LLVM doesn't seem to "like" large
functions, as a
>> general rule. Admittedly, my experience is similar with gcc, so I'm
not
>> sure it's something that can be easily fixed. And I'm probably
sounding
>> like a broken record, because I have said this before.
>>
>> My experience is that the time it takes to compile something is growing
>> above linear with size of function.
>>
>>
> The number of BBs -- Kosyia can point you to the compile time bug that is
> exposed by asan .
>
> https://llvm.org/bugs/show_bug.cgi?id=17409
Not just asan, this bug reproduces in a wide range of cases.
By now I am not even sure if this is a single bug or a set of independent
(but similarly looking) problems.


> David
>
>
>
>> Of course, the LLVM code is growing over time, both to support more
>> features and to support more architectures, new processor types and
>> instruction sets, at least of which will lead to larger functions in
>> general [and this is the function "after inlining", so
splitting small
>> 'called once' functions out doesn't really help much].
>>
>> I will have a little play to see if I can identify more of a cuplrit
[at
>> the very least if it's "large basic blocks" or
"large functions" that is
>> the problem] - of course, this could be unrelated and irellevant to the
>> problem Daniel is pointing at, and it may or may not be easily
resolved...
>>
>> --
>> Mats
>>
>> On 8 March 2016 at 18:42, Richard Smith via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> On Tue, Mar 8, 2016 at 8:13 AM, Rafael Espíndola
>>> <llvm-dev at lists.llvm.org> wrote:
>>> > I have just benchmarked building trunk llvm and clang in
Debug,
>>> > Release and LTO modes (see the attached scrip for the cmake
lines).
>>> >
>>> > The compilers used were clang 3.5, 3.6, 3.7, 3.8 and trunk. In
all
>>> > cases I used the system libgcc and libstdc++.
>>> >
>>> > For release builds there is a monotonic increase in each
version. From
>>> > 163 minutes with 3.5 to 212 minutes with trunk. For
comparison, gcc
>>> > 5.3.2 takes 205 minutes.
>>> >
>>> > Debug and LTO show an improvement in 3.7, but have regressed
again in
>>> 3.8.
>>>
>>> I'm curious how these times divide across Clang and various
parts of
>>> LLVM; rerunning with -ftime-report and summing the numbers across
all
>>> compiles could be interesting.
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160309/23440ea1/attachment.html>

A historical note:

Back in the pre-Clang LLVM 1.x dark ages you could, if you
pressed the right buttons, run LLVM as a very fast portable
codegen. MB/s was a reasonable measure as the speed was (or
could be made to be) fairly independent of the input structure.

Since ~2006, as LLVM has shifted from "awesome research
plaything" to "compiler people depend on", there has been a
focus on ensuring that typical software compiles quickly and
well. Many good things have followed as a result, but you are
certainly correct that LLVM doesn't handle large input
particularly well. Having said that, some projects (the Gambit
Scheme->C and Verilator Verilog->C compilers come to mind)
routinely see runtimes 10~100x that of GCC in typical use. So
perhaps we are thinking of different things if you're seeing
similar issues with GCC.

I suspect that despite the passage of time the problem remains
solvable - there's probably *more* work to be done now, but I
don't think there are any massively *difficult* problems to be
solved. Properly quantifying/tracking the problem would be a
good first step.

Best,
Duraid


On Tue, Mar 08, 2016 at 06:55:24PM +0000, mats petersson via llvm-dev
wrote:
> I have noticed that LLVM doesn't seem to "like" large
functions, as a
> general rule. Admittedly, my experience is similar with gcc, so I'm not
> sure it's something that can be easily fixed. And I'm probably
sounding
> like a broken record, because I have said this before.
> 
> My experience is that the time it takes to compile something is growing
> above linear with size of function.
> 
> Of course, the LLVM code is growing over time, both to support more
> features and to support more architectures, new processor types and
> instruction sets, at least of which will lead to larger functions in
> general [and this is the function "after inlining", so splitting
small
> 'called once' functions out doesn't really help much].
> 
> I will have a little play to see if I can identify more of a cuplrit [at
> the very least if it's "large basic blocks" or "large
functions" that is
> the problem] - of course, this could be unrelated and irellevant to the
> problem Daniel is pointing at, and it may or may not be easily resolved...
> 
> --
> Mats
> 
> On 8 March 2016 at 18:42, Richard Smith via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> 
> > On Tue, Mar 8, 2016 at 8:13 AM, Rafael Espíndola
> > <llvm-dev at lists.llvm.org> wrote:
> > > I have just benchmarked building trunk llvm and clang in Debug,
> > > Release and LTO modes (see the attached scrip for the cmake
lines).
> > >
> > > The compilers used were clang 3.5, 3.6, 3.7, 3.8 and trunk. In
all
> > > cases I used the system libgcc and libstdc++.
> > >
> > > For release builds there is a monotonic increase in each version.
From
> > > 163 minutes with 3.5 to 212 minutes with trunk. For comparison,
gcc
> > > 5.3.2 takes 205 minutes.
> > >
> > > Debug and LTO show an improvement in 3.7, but have regressed
again in
> > 3.8.
> >
> > I'm curious how these times divide across Clang and various parts
of
> > LLVM; rerunning with -ftime-report and summing the numbers across all
> > compiles could be interesting.
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

> On Mar 9, 2016, at 3:57 PM, Duraid Madina via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> A historical note:
> 
> Back in the pre-Clang LLVM 1.x dark ages you could, if you
> pressed the right buttons, run LLVM as a very fast portable
> codegen. MB/s was a reasonable measure as the speed was (or
> could be made to be) fairly independent of the input structure.
> 
> Since ~2006, as LLVM has shifted from "awesome research
> plaything" to "compiler people depend on", there has been a
> focus on ensuring that typical software compiles quickly and
> well. Many good things have followed as a result, but you are
> certainly correct that LLVM doesn't handle large input
> particularly well. Having said that, some projects (the Gambit
> Scheme->C and Verilator Verilog->C compilers come to mind)
> routinely see runtimes 10~100x that of GCC in typical use. So
> perhaps we are thinking of different things if you're seeing
> similar issues with GCC.
Bug reports (with pre-processed source files preferably) are always welcome.

Collecting the test cases in a "compile time test suite" is what
should follow naturally.

Best,

-- 
Mehdi


> 
> I suspect that despite the passage of time the problem remains
> solvable - there's probably *more* work to be done now, but I
> don't think there are any massively *difficult* problems to be
> solved. Properly quantifying/tracking the problem would be a
> good first step.
> 
> Best,
> Duraid
> 
> 
> On Tue, Mar 08, 2016 at 06:55:24PM +0000, mats petersson via llvm-dev
wrote:
>> I have noticed that LLVM doesn't seem to "like" large
functions, as a
>> general rule. Admittedly, my experience is similar with gcc, so I'm
not
>> sure it's something that can be easily fixed. And I'm probably
sounding
>> like a broken record, because I have said this before.
>> 
>> My experience is that the time it takes to compile something is growing
>> above linear with size of function.
>> 
>> Of course, the LLVM code is growing over time, both to support more
>> features and to support more architectures, new processor types and
>> instruction sets, at least of which will lead to larger functions in
>> general [and this is the function "after inlining", so
splitting small
>> 'called once' functions out doesn't really help much].
>> 
>> I will have a little play to see if I can identify more of a cuplrit
[at
>> the very least if it's "large basic blocks" or
"large functions" that is
>> the problem] - of course, this could be unrelated and irellevant to the
>> problem Daniel is pointing at, and it may or may not be easily
resolved...
>> 
>> --
>> Mats
>> 
>> On 8 March 2016 at 18:42, Richard Smith via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>> 
>>> On Tue, Mar 8, 2016 at 8:13 AM, Rafael Espíndola
>>> <llvm-dev at lists.llvm.org> wrote:
>>>> I have just benchmarked building trunk llvm and clang in Debug,
>>>> Release and LTO modes (see the attached scrip for the cmake
lines).
>>>> 
>>>> The compilers used were clang 3.5, 3.6, 3.7, 3.8 and trunk. In
all
>>>> cases I used the system libgcc and libstdc++.
>>>> 
>>>> For release builds there is a monotonic increase in each
version. From
>>>> 163 minutes with 3.5 to 212 minutes with trunk. For comparison,
gcc
>>>> 5.3.2 takes 205 minutes.
>>>> 
>>>> Debug and LTO show an improvement in 3.7, but have regressed
again in
>>> 3.8.
>>> 
>>> I'm curious how these times divide across Clang and various
parts of
>>> LLVM; rerunning with -ftime-report and summing the numbers across
all
>>> compiles could be interesting.
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> 
> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev