llvm dev - Sep 2013 - [LLVMdev] RFC - Extending ProfileInfo to support external profiles

Sent from my iPad

On Sep 6, 2013, at 10:01 AM, Chandler Carruth <chandlerc at google.com>
wrote:
> 
> On Fri, Sep 6, 2013 at 6:00 AM, Diego Novillo <dnovillo at
google.com> wrote:
>> Is my understanding more or less correct?
> 
> Probably for all I know. I'm not aware of anyone actively maintaining
or keeping these pieces working. AFAICT, there have been 3-4 projects that
started hacking on this and didn't finish, and frankly the result is a mess.
It's not clear than any of it really works today.
>  
>>  If so, I would like to
>> make three main changes:
>> 
>> 1- Unify profile loading so that it only talks to the basic
>>    analysis API. Here, I would like to change the way that
>>    profile information is reflected in the program. Currently,
>>    the compiler loads data from the profile file into 4 arrays
>>    inside ProfileInfo.  This is then reflected into the CFG for
>>    the corresponding function.
>> 
>>    Instead, I would like to reflect profile info as metadata in
>>    the IR. For instance, functions seldom called in the profile
>>    could be annotated with the 'cold' attribute in the IR.
Block
>>    frequency is more straightforward: we add the frequency data
>>    to the first instruction of the block.
>> 
>>    Things like that. The problem here is that different profiling
>>    mechanisms will have different kinds of data. We will need
>>    ways of translating those into metadata that is then served
>>    from the analysis API. For example, how do we deal with value
>>    profilers?
>> 
>>    For now, I'm focusing on the most common types of profilers.
>>    These tend to use block/edge frequencies, which are more
>>    straightforward to convert.
> 
> I really think you should ignore the existing profile support in LLVM. ALl
of it. I actually think we should remove all of it, but that can happen later.
I haven't look at the profiling support recently. But this doesn't quite
match my understanding. Bob, do you have comments?

Evan
> 
> I would build whatever passes and infrastructure you need in order to load
your profile data, in whatever format makes sense, and annotate the IR with
metadata. I would just build them from scratch. I don't think there is
anything to be gained from trying toclean up the existing profiling
infrastructure when you'll end up re-using almost none of it, you will have
different constraints when dealing with profiles from sampling tools, and will
still have a common IR-level interface in the metadata.
>  
>> 
>> 2- Passes should not need to be aware that profile
>>    information exists. They would make the usual calls to the
>>    analysis API, which will use profile data, if available.
>> 
>>    With this, I want to make optimizers automatically make
>>    better decisions when profile information exists.
> 
> I believe this has already been done. I'm not aware of any optimization
passes in use today that will need changes here.
>  
>> 
>> 3- Profile data should be flexible wrt code drift. This is
>>    something that the profile loader should deal with. It
>>    currently fails when the profile is out of sync with the shape
>>    of the IR. In the presence of stale profile:
>>    (a) Do not fail hard.
>>    (b) Make conservative decisions with the annotations it
>>        generates.
>>    (c) Provide feedback to the user about the staleness of the
>>        profile (e.g. % of samples/records dropped from the profile).
> 
> Yes, in designing your own (from the ground up) loader, these will be
important issues.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130906/db392751/attachment.html>

On Sep 6, 2013, at 8:43 PM, Evan Cheng <evan.cheng at apple.com> wrote:
> 
> 
> Sent from my iPad
> 
> On Sep 6, 2013, at 10:01 AM, Chandler Carruth <chandlerc at
google.com> wrote:
> 
>> 
>> On Fri, Sep 6, 2013 at 6:00 AM, Diego Novillo <dnovillo at
google.com> wrote:
>> Is my understanding more or less correct?
>> 
>> Probably for all I know. I'm not aware of anyone actively
maintaining or keeping these pieces working. AFAICT, there have been 3-4
projects that started hacking on this and didn't finish, and frankly the
result is a mess. It's not clear than any of it really works today.
>>  
>>  If so, I would like to
>> make three main changes:
>> 
>> 1- Unify profile loading so that it only talks to the basic
>>    analysis API. Here, I would like to change the way that
>>    profile information is reflected in the program. Currently,
>>    the compiler loads data from the profile file into 4 arrays
>>    inside ProfileInfo.  This is then reflected into the CFG for
>>    the corresponding function.
>> 
>>    Instead, I would like to reflect profile info as metadata in
>>    the IR. For instance, functions seldom called in the profile
>>    could be annotated with the 'cold' attribute in the IR.
Block
>>    frequency is more straightforward: we add the frequency data
>>    to the first instruction of the block.
>> 
>>    Things like that. The problem here is that different profiling
>>    mechanisms will have different kinds of data. We will need
>>    ways of translating those into metadata that is then served
>>    from the analysis API. For example, how do we deal with value
>>    profilers?
>> 
>>    For now, I'm focusing on the most common types of profilers.
>>    These tend to use block/edge frequencies, which are more
>>    straightforward to convert.
>> 
>> I really think you should ignore the existing profile support in LLVM.
ALl of it. I actually think we should remove all of it, but that can happen
later.
> 
> I haven't look at the profiling support recently. But this doesn't
quite match my understanding. Bob, do you have comments?
> 
> Evan
I agree with Chandler.  The metadata-level branch weights, block frequencies,
etc. are all reasonable, but the profiling stuff is pretty rough.  It has been
useful for some research projects, and we’ve been able to use it for some
experiments but I don’t think it makes sense to use it at all for what Diego is
proposing.
> 
>> 
>> I would build whatever passes and infrastructure you need in order to
load your profile data, in whatever format makes sense, and annotate the IR with
metadata. I would just build them from scratch. I don't think there is
anything to be gained from trying toclean up the existing profiling
infrastructure when you'll end up re-using almost none of it, you will have
different constraints when dealing with profiles from sampling tools, and will
still have a common IR-level interface in the metadata.
>>  
>> 
>> 2- Passes should not need to be aware that profile
>>    information exists. They would make the usual calls to the
>>    analysis API, which will use profile data, if available.
>> 
>>    With this, I want to make optimizers automatically make
>>    better decisions when profile information exists.
>> 
>> I believe this has already been done. I'm not aware of any
optimization passes in use today that will need changes here.
>>  
>> 
>> 3- Profile data should be flexible wrt code drift. This is
>>    something that the profile loader should deal with. It
>>    currently fails when the profile is out of sync with the shape
>>    of the IR. In the presence of stale profile:
>>    (a) Do not fail hard.
>>    (b) Make conservative decisions with the annotations it
>>        generates.
>>    (c) Provide feedback to the user about the staleness of the
>>        profile (e.g. % of samples/records dropped from the profile).
>> 
>> Yes, in designing your own (from the ground up) loader, these will be
important issues.
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130906/65fdf048/attachment.html>

Sent from my iPad

On Sep 6, 2013, at 9:43 PM, Bob Wilson <bob.wilson at apple.com> wrote:
> On Sep 6, 2013, at 8:43 PM, Evan Cheng <evan.cheng at apple.com>
wrote:
> 
>> 
>> 
>> Sent from my iPad
>> 
>> On Sep 6, 2013, at 10:01 AM, Chandler Carruth <chandlerc at
google.com> wrote:
>> 
>>> 
>>> On Fri, Sep 6, 2013 at 6:00 AM, Diego Novillo <dnovillo at
google.com> wrote:
>>>> Is my understanding more or less correct?
>>> 
>>> Probably for all I know. I'm not aware of anyone actively
maintaining or keeping these pieces working. AFAICT, there have been 3-4
projects that started hacking on this and didn't finish, and frankly the
result is a mess. It's not clear than any of it really works today.
>>>  
>>>>  If so, I would like to
>>>> make three main changes:
>>>> 
>>>> 1- Unify profile loading so that it only talks to the basic
>>>>    analysis API. Here, I would like to change the way that
>>>>    profile information is reflected in the program. Currently,
>>>>    the compiler loads data from the profile file into 4 arrays
>>>>    inside ProfileInfo.  This is then reflected into the CFG for
>>>>    the corresponding function.
>>>> 
>>>>    Instead, I would like to reflect profile info as metadata in
>>>>    the IR. For instance, functions seldom called in the profile
>>>>    could be annotated with the 'cold' attribute in the
IR. Block
>>>>    frequency is more straightforward: we add the frequency data
>>>>    to the first instruction of the block.
>>>> 
>>>>    Things like that. The problem here is that different
profiling
>>>>    mechanisms will have different kinds of data. We will need
>>>>    ways of translating those into metadata that is then served
>>>>    from the analysis API. For example, how do we deal with
value
>>>>    profilers?
>>>> 
>>>>    For now, I'm focusing on the most common types of
profilers.
>>>>    These tend to use block/edge frequencies, which are more
>>>>    straightforward to convert.
>>> 
>>> I really think you should ignore the existing profile support in
LLVM. ALl of it. I actually think we should remove all of it, but that can
happen later.
>> 
>> I haven't look at the profiling support recently. But this
doesn't quite match my understanding. Bob, do you have comments?
>> 
>> Evan
> 
> I agree with Chandler.  The metadata-level branch weights, block
frequencies, etc. are all reasonable, but the profiling stuff is pretty rough. 
It has been useful for some research projects, and we’ve been able to use it for
some experiments but I don’t think it makes sense to use it at all for what
Diego is proposing.
Ok, I see what you guys are referring to. Sorry for the noise. 

Evan
> 
>> 
>>> 
>>> I would build whatever passes and infrastructure you need in order
to load your profile data, in whatever format makes sense, and annotate the IR
with metadata. I would just build them from scratch. I don't think there is
anything to be gained from trying toclean up the existing profiling
infrastructure when you'll end up re-using almost none of it, you will have
different constraints when dealing with profiles from sampling tools, and will
still have a common IR-level interface in the metadata.
>>>  
>>>> 
>>>> 2- Passes should not need to be aware that profile
>>>>    information exists. They would make the usual calls to the
>>>>    analysis API, which will use profile data, if available.
>>>> 
>>>>    With this, I want to make optimizers automatically make
>>>>    better decisions when profile information exists.
>>> 
>>> I believe this has already been done. I'm not aware of any
optimization passes in use today that will need changes here.
>>>  
>>>> 
>>>> 3- Profile data should be flexible wrt code drift. This is
>>>>    something that the profile loader should deal with. It
>>>>    currently fails when the profile is out of sync with the
shape
>>>>    of the IR. In the presence of stale profile:
>>>>    (a) Do not fail hard.
>>>>    (b) Make conservative decisions with the annotations it
>>>>        generates.
>>>>    (c) Provide feedback to the user about the staleness of the
>>>>        profile (e.g. % of samples/records dropped from the
profile).
>>> 
>>> Yes, in designing your own (from the ground up) loader, these will
be important issues.
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130906/6751ce8e/attachment.html>