similar to: Add support for in-process profile merging in profile-runtime

I have thought about this issue too, in the context of games. We may want to turn profiling only for certain frames (essentially, this is many small profile runs). However, I have not seen it demonstrated that this kind of refined data collection will actually improve PGO results in practice. The evidence I do have though is that IIRC Apple have found that almost all of the benefits of PGO for

> On Feb 28, 2016, at 12:46 AM, Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Justin, looks like there is some misunderstanding in my email. I want to clarify it here first: > > 1) I am not proposing changing the default profile dumping model as used today. The online merging is totally optional; > 2) the on-line profile merging is not doing

On Sat, Feb 27, 2016 at 6:50 PM, Sean Silva <chisophugis at gmail.com> wrote: > I have thought about this issue too, in the context of games. We may want > to turn profiling only for certain frames (essentially, this is many small > profile runs). > > However, I have not seen it demonstrated that this kind of refined data > collection will actually improve PGO results in

Justin, looks like there is some misunderstanding in my email. I want to clarify it here first: 1) I am not proposing changing the default profile dumping model as used today. The online merging is totally optional; 2) the on-line profile merging is not doing conversion from raw to index format. It does very simple raw-to-raw merging using existing runtime APIs. 3) the change to existing profile

----- Original Message ----- > From: "Sean Silva via llvm-dev" <llvm-dev at lists.llvm.org> > To: "Xinliang David Li" <davidxl at google.com> > Cc: "llvm-dev" <llvm-dev at lists.llvm.org> > Sent: Saturday, February 27, 2016 8:50:05 PM > Subject: Re: [llvm-dev] Add support for in-process profile merging in profile-runtime > >

+ 1 to Sean's suggestion of using a wrapper script to call profdata merge. David, does that work for your use case? Some inline comments --- > On Feb 28, 2016, at 10:45 AM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: > >> >> On Feb 28, 2016, at 12:46 AM, Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >>

Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> writes: > One of the main missing features in Clang/LLVM profile runtime is the lack of > support for online/in-process profile merging support. Profile data collected > for different workloads for the same executable binary need to be collected > and merged later by the offline post-processing tool.  This limitation

Hi David, This is wonderful data and demonstrates the viability of this feature. I think this has alleviated the concerns regarding file locking. As far as the implementation of the feature, I think we will probably want the following incremental steps: a) implement the core merging logic and add to buffer API a primitive for merging two buffers b) implement the file system glue to extend this

Hi All, Our team at Facebook is building a new context-sensitive Sample PGO as an alternative to the existing AutoFDO. We’d like to share our motivation, propose a new design, and reveal preliminary results on benchmarks. We will refer to the proposed design as CSSPGO in this RFC. The new CSSPGO leverages simultaneous LBR and stack sampling to construct a full context-sensitive profile. It

Hi Diego, thanks for clarifying the difference between the two formats. I have noticed the new note in the "Sample Profile Format" section of the Clang guide clarifying that it is different from the coverage format. So, my further question is... Am I right in understanding that both formats can be used for PGO purposes then? I have tried the following, as in the Clang user guide: $

Hi David, Thanks for your reply. I find that I need to add some new types of profile data that are specific to JIT environment like Function Ordering. Function Ordering is similar to dynamic call graph which records the execution of functions at runtime along with the order in which they are called. Eg: Suppose they are 5 functions (F1..F5). F1 calls other functions in the order described

Problem ------- Instrumentation for PGO and frontend-based coverage places a large amount of data in object files, even though the majority of this data is not needed at run-time. All the data is needlessly duplicated while generating archives, and again while linking. PGO name data is written out into raw profiles by instrumented programs, slowing down the training and code coverage workflows.

See my answers inline. From: Xinliang David Li <davidxl at google.com> Date: Friday, August 7, 2020 at 7:57 PM To: Wenlei He <wenlei at fb.com> Cc: "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>, Wei Mi <wmi at google.com>, Hongtao Yu <hoy at fb.com> Subject: Re: [RFC] Context-sensitive Sample PGO with Pseudo-Instrumentation On Fri, Aug 7,

On Fri, Mar 4, 2016 at 11:28 AM, Tilmann Scheller <tilmann at osg.samsung.com> wrote: > Hi Sedat, > > On 03/03/2016 08:09 AM, Sedat Dilek via llvm-dev wrote: >> >> It might be that a CLANG generated with LTO/PGO speeds up the build. >> Can you confirm this? > > Yes, a Clang host compiler built with LTO or PGO is generally faster than an > -O3 build. >

On Fri, Jun 30, 2017 at 10:04 PM, Sean Silva <chisophugis at gmail.com> wrote: > > > On Fri, Jun 30, 2017 at 5:54 PM, via llvm-dev <llvm-dev at lists.llvm.org> > wrote: > >> Problem >> ------- >> >> Instrumentation for PGO and frontend-based coverage places a large amount >> of >> data in object files, even though the majority of this

Thanks for the thoughtful questions, David. See my answers inline. Thanks, Wenlei From: Xinliang David Li <davidxl at google.com> Date: Friday, August 7, 2020 at 1:24 PM To: Wenlei He <wenlei at fb.com> Cc: "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>, Wei Mi <wmi at google.com>, Hongtao Yu <hoy at fb.com> Subject: Re: [RFC] Context-sensitive

On Fri, Aug 7, 2020 at 11:28 PM Hongtao Yu <hoy at fb.com> wrote: > A few add-ons. > > > > *From: *Wenlei He <wenlei at fb.com> > *Date: *Friday, August 7, 2020 at 10:34 PM > *To: *Xinliang David Li <davidxl at google.com> > *Cc: *"llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>, Wei Mi < > wmi at

Also see my replies inline. From: Hongtao Yu <hoy at fb.com> Date: Saturday, August 8, 2020 at 11:25 AM To: Xinliang David Li <davidxl at google.com> Cc: Wenlei He <wenlei at fb.com>, "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>, Wei Mi <wmi at google.com> Subject: Re: [RFC] Context-sensitive Sample PGO with Pseudo-Instrumentation Replied

On Fri, Jun 30, 2017 at 5:54 PM, via llvm-dev <llvm-dev at lists.llvm.org> wrote: > Problem > ------- > > Instrumentation for PGO and frontend-based coverage places a large amount > of > data in object files, even though the majority of this data is not needed > at > run-time. All the data is needlessly duplicated while generating archives, > and > again while

On Fri, May 22, 2015 at 11:16 AM, Dario Domizioli <dario.domizioli at gmail.com> wrote: > Hi all, > > I am a bit confused about the documentation of the format of the profile > data file. > > The Clang user guide here describes it as an ASCII text file: > http://clang.llvm.org/docs/UsersManual.html#sample-profile-format > > Whereas the posts above and the