llvm dev - Jun 2015 - [LLVMdev] RFC - Stop ignoring -fprofile-generate and -fprofile-use

On 2015 Jun 17, at 13:53, Diego Novillo <dnovillo at google.com>
wrote:
> The flags -fprofile-generate and -fprofile-use are currently ignored
> for GCC compatibility.  I would like to enable them and give them
> similar semantics to GCC.  These flags are baked pretty deeply into
> our build environment, so supporting them at the driver level will
> make our lives a lot simpler.
This seems like a fairly reasonable idea. We'll have to make sure it's
clear (in the docs or whatever) that you can't use clang to generate a
profile and gcc to consume it and vice versa, but that doesn't seem like
a big deal.
> From https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html:
> 
> -fprofile-generate
> -fprofile-generate=path
> Enable options usually used for instrumenting application to produce
> profile useful for later recompilation with profile feedback based
> optimization. You must use -fprofile-generate both when compiling
> and when linking your program.
> The following options are enabled: -fprofile-arcs, -fprofile-values, -fvpt.
> 
> If path is specified, GCC looks at the path to find the profile
> feedback data files. See -fprofile-dir.
> 
> -fprofile-use
> -fprofile-use=path
> Enable profile feedback-directed optimizations, and the following
> optimizations which are generally profitable only with profile
> feedback available: -fbranch-probabilities, -fvpt, -funroll-loops,
> -fpeel-loops, -ftracer, -ftree-vectorize, and
> ftree-loop-distribute-patterns.
> By default, GCC emits an error message if the feedback profiles do
> not match the source code. This error can be turned into a warning
> by using -Wcoverage-mismatch. Note this may result in poorly
> optimized code.
> 
> If path is specified, GCC looks at the path to find the profile
> feedback data files. See -fprofile-dir.
> 
> 
> Note that the argument to -fprofile-generate and -fprofile-use is
> not a file name. It is a path prefix used to store the generated
> profile (in GCC's case, each object file in the binary generates its
> own profile file). In Clang, the flags would do the following:
>
>   • -fprofile-generate=path-prefix would cause the instrumented
>     binary to write the profile <path-prefix>/default.profraw. If
>     <path-prefix> does not exist, it is created by the runtime.
>   • -fprofile-use=path-prefix would cause the compiler to read
>     from <path-prefix>/default.profile.
The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option. I'd limit
this to the path-to-a-file case though, I think it's too magical when
pointing at a directory.
>   • I could also add support for -fprofile-dir, but we don't
>     really use it internally.
I don't think it's necessary unless someone actually wants to use it.
> As with -fprofile-instr-generate, LLVM_PROFILE_FILE would override
> the path prefix and name of the profile file.
> 
> Does this sound reasonable?
> 
> 
> Thanks.  Diego.

> The path-prefix versions are a bit of an odd fit for the instrprof
> stuff - I'd prefer if they dealt with the path to the file. I can see
> how that might be difficult to do and still be compatible though.
>
> Maybe we could make the -fprofile-use= version accept a file or
> directory, and if a directory is specified then look for the default?
That sounds reasonable.
>
> The other thing that would be nice would be if -fprofile-use=path could
> autodetect which kind of profile the path is - that is, It'd be nice to
> accept both instrprof and sample profiles with this option. I'd limit
> this to the path-to-a-file case though, I think it's too magical when
> pointing at a directory.
Agree.

David
>
>>   • I could also add support for -fprofile-dir, but we don't
>>     really use it internally.
>
> I don't think it's necessary unless someone actually wants to use
it.
>
>> As with -fprofile-instr-generate, LLVM_PROFILE_FILE would override
>> the path prefix and name of the profile file.
>>
>> Does this sound reasonable?
>>
>>
>> Thanks.  Diego.

On Wed, Jun 17, 2015 at 5:34 PM, Justin Bogner <mail at justinbogner.com>
wrote:
> On 2015 Jun 17, at 13:53, Diego Novillo <dnovillo at google.com>
wrote:
>
> >   • -fprofile-generate=path-prefix would cause the instrumented
> >     binary to write the profile <path-prefix>/default.profraw.
If
> >     <path-prefix> does not exist, it is created by the runtime.
> >   • -fprofile-use=path-prefix would cause the compiler to read
> >     from <path-prefix>/default.profile.
>
> The path-prefix versions are a bit of an odd fit for the instrprof
> stuff - I'd prefer if they dealt with the path to the file. I can see
> how that might be difficult to do and still be compatible though.
>
> Maybe we could make the -fprofile-use= version accept a file or
> directory, and if a directory is specified then look for the default?
>
Sure. Sounds reasonable.

> The other thing that would be nice would be if -fprofile-use=path could
> autodetect which kind of profile the path is - that is, It'd be nice to
> accept both instrprof and sample profiles with this option.

Sure.  But this may have a wrinkle. The problem here is that GCC already
has a different option for sample profiles (-fauto-profile). Someone using
the same flags for both compilers (or migrating off of GCC) might find this
problematic. Although, I can't really think a scenario where this would
happen.

In a sense, anything that is a superset of GCC's current behaviour should
be fine. So, I suppose this is OK.


> I'd limit
> this to the path-to-a-file case though, I think it's too magical when
> pointing at a directory.
>
Agreed.


Diego.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150617/560f3435/attachment.html>

On Wed, Jun 17, 2015 at 3:07 PM, Diego Novillo <dnovillo at google.com>
wrote:
>
>
> On Wed, Jun 17, 2015 at 5:34 PM, Justin Bogner <mail at
justinbogner.com>
> wrote:
>>
>> On 2015 Jun 17, at 13:53, Diego Novillo <dnovillo at google.com>
wrote:
>>
>> >   • -fprofile-generate=path-prefix would cause the instrumented
>> >     binary to write the profile
<path-prefix>/default.profraw. If
>> >     <path-prefix> does not exist, it is created by the
runtime.
>> >   • -fprofile-use=path-prefix would cause the compiler to read
>> >     from <path-prefix>/default.profile.
>>
>> The path-prefix versions are a bit of an odd fit for the instrprof
>> stuff - I'd prefer if they dealt with the path to the file. I can
see
>> how that might be difficult to do and still be compatible though.
>>
>> Maybe we could make the -fprofile-use= version accept a file or
>> directory, and if a directory is specified then look for the default?
>
>
> Sure. Sounds reasonable.
>
>>
>> The other thing that would be nice would be if -fprofile-use=path could
>> autodetect which kind of profile the path is - that is, It'd be
nice to
>> accept both instrprof and sample profiles with this option.
>
>
> Sure.  But this may have a wrinkle. The problem here is that GCC already
has
> a different option for sample profiles (-fauto-profile). Someone using the
> same flags for both compilers (or migrating off of GCC) might find this
> problematic. Although, I can't really think a scenario where this would
> happen.
>
> In a sense, anything that is a superset of GCC's current behaviour
should be
> fine. So, I suppose this is OK.
>
yes -- you are not removing the old behavior.
>
>>
>> I'd limit
>> this to the path-to-a-file case though, I think it's too magical
when
>> pointing at a directory.
>
>
> Agreed.
>
>
> Diego.

Diego Novillo <dnovillo at google.com> writes:
> On Wed, Jun 17, 2015 at 5:34 PM, Justin Bogner <mail at
justinbogner.com> wrote:
>> On 2015 Jun 17, at 13:53, Diego Novillo <dnovillo at google.com>
wrote:
>>>   • -fprofile-generate=path-prefix would cause the instrumented
>>>     binary to write the profile
<path-prefix>/default.profraw. If
>>>     <path-prefix> does not exist, it is created by the
runtime.
>>>   • -fprofile-use=path-prefix would cause the compiler to read
>>>     from <path-prefix>/default.profile.
>>
>> The path-prefix versions are a bit of an odd fit for the instrprof
>> stuff - I'd prefer if they dealt with the path to the file. I can
see
>> how that might be difficult to do and still be compatible though.
>>
>> Maybe we could make the -fprofile-use= version accept a file or
>> directory, and if a directory is specified then look for the default?
>
> Sure. Sounds reasonable.
>
>> The other thing that would be nice would be if -fprofile-use=path could
>> autodetect which kind of profile the path is - that is, It'd be
nice to
>> accept both instrprof and sample profiles with this option.
>
> Sure.  But this may have a wrinkle. The problem here is that GCC already
has a
> different option for sample profiles (-fauto-profile). Someone using the
same
> flags for both compilers (or migrating off of GCC) might find this
> problematic. Although, I can't really think a scenario where this would
> happen.
Convince gcc to do this too? ;)

I'm not terribly attached to this idea if you think it's a bad precedent
to set or something, but it does seem like it'd be convenient enough to
be worthwhile.
> In a sense, anything that is a superset of GCC's current behaviour
should be
> fine. So, I suppose this is OK.