llvm dev - Jul 2013 - [LLVMdev] [LLVM Dev] [Discussion] Function-based parallel LLVM backend code generation

On Tue, Jul 16, 2013 at 1:18 PM, Xinliang David Li <xinliangli at
gmail.com>wrote:
> Ignoring FE time which can be fully parallelized and assuming 10%
> compile time is spent in serial module passes, 25% time is spent in
> CGSCC pass, the maximum speed up that can be gained by using function
> level parallelism is less than 3x.  Even adding support for parallel
> compilation for leaves of CG in CGSCC pass won't help too much -- the
> percentage of leaf functions is < 30% in large apps I have seen.
>
Can you clarify what you're basing these assumption on or how you derived
your data?

> Module based parallelism proposed by Shuxin has max speed up of 10x,
> assuming body cloning does not add a lot overhead and build farm with
> hundred/thousands of nodes is used.
>
Body cloning does add some overhead, so that actually needs to be measured.
Also, many don't have such a build farm.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130716/c3f73c10/attachment.html>

On Tue, Jul 16, 2013 at 1:33 PM, Chandler Carruth <chandlerc at
google.com> wrote:
> On Tue, Jul 16, 2013 at 1:18 PM, Xinliang David Li <xinliangli at
gmail.com>
> wrote:
>>
>> Ignoring FE time which can be fully parallelized and assuming 10%
>> compile time is spent in serial module passes, 25% time is spent in
>> CGSCC pass, the maximum speed up that can be gained by using function
>> level parallelism is less than 3x.  Even adding support for parallel
>> compilation for leaves of CG in CGSCC pass won't help too much --
the
>> percentage of leaf functions is < 30% in large apps I have seen.
>
>
> Can you clarify what you're basing these assumption on or how you
derived
> your data?
>
Those numbers are purely speculative -- does Clang has an option to
dump the time breakout of each passes such as -ftime-report in GCC?

thanks,

David
>>
>> Module based parallelism proposed by Shuxin has max speed up of 10x,
>> assuming body cloning does not add a lot overhead and build farm with
>> hundred/thousands of nodes is used.
>
>
> Body cloning does add some overhead, so that actually needs to be measured.
> Also, many don't have such a build farm.

On Tue, Jul 16, 2013 at 1:37 PM, Xinliang David Li <xinliangli at
gmail.com>wrote:
> On Tue, Jul 16, 2013 at 1:33 PM, Chandler Carruth <chandlerc at
google.com>
> wrote:
> > On Tue, Jul 16, 2013 at 1:18 PM, Xinliang David Li <xinliangli at
gmail.com
> >
> > wrote:
> >>
> >> Ignoring FE time which can be fully parallelized and assuming 10%
> >> compile time is spent in serial module passes, 25% time is spent
in
> >> CGSCC pass, the maximum speed up that can be gained by using
function
> >> level parallelism is less than 3x.  Even adding support for
parallel
> >> compilation for leaves of CG in CGSCC pass won't help too much
-- the
> >> percentage of leaf functions is < 30% in large apps I have
seen.
> >
> >
> > Can you clarify what you're basing these assumption on or how you
derived
> > your data?
> >
>
> Those numbers are purely speculative -- does Clang has an option to
> dump the time breakout of each passes such as -ftime-report in GCC?
>
We have the functionality... I thought we wired -ftime-report up to it? If
that doesn't work I'll have to go digging.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130716/7c08f027/attachment.html>