llvm dev - Jul 2020 - [profile] Heads Up: Please Double Check the Instrument Level of Your PGO Profile Files

Hi All,

Recently we discovered a (hidden) performance bug that happened when you’re
using Full LTO setup + BFD linker to compile first-stage PGO executables that
were expected to generate IR level profile files. That is, compiling an
executable using following commands:
```
clang -flto -fprofile-generate -c sample.c -o sample.c.bc.o
clang -flto sample.c.bc.o -o sample_exe -fprofile-generate
```
The  resulting sample_exe executable will always emit Frontend level profile
files despite the presence of the -fprofile-generate flag, which should make the
executable generate IR level profile files. Other prerequisites to reproduce
this problem including:
	• Using (latest version of) BFD linker
	• Using LLVM’s own LTO linker plugin (i.e. LLVMgold.so)
	• Using LLVM’s compiler-rt as the runtime library
	• On a Linux / BSD platform
This problem has been confirmed not to happen with gold and LLD linkers. Impacts
on usages on Windows platform remain unknown.

I just cooked a patch for this bug: https://reviews.llvm.org/D83967
<https://reviews.llvm.org/D83967>
Basically the solution (and the root cause) is similar to this patch several
years ago: https://reviews.llvm.org/D34797
<https://reviews.llvm.org/D34797>

The intriguing (or annoying) part of this bug is that you’ll never find the
issue (since it neither crashes nor causes significant performance regression)
unless you take a look into the generated profile files. So I just want to put a
heads up here, though I know most of the folks here are moving away from using
BFD linkers.

Best,
-Min
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200716/e791ec4f/attachment-0001.html>

This sounds a lot like https://bugs.llvm.org/show_bug.cgi?id=45050. I'm
glad to hear you figured out the root cause, thank you!


On Thu, Jul 16, 2020 at 1:55 PM Min-Yih Hsu via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi All,
>
> Recently we discovered a (hidden) performance bug that happened when
> you’re using Full LTO setup + BFD linker to compile first-stage PGO
> executables that were expected to generate IR level profile files. That is,
> compiling an executable using following commands:
> ```
> clang -flto -fprofile-generate -c sample.c -o sample.c.bc.o
> clang -flto sample.c.bc.o -o sample_exe -fprofile-generate
> ```
> The  resulting sample_exe executable will *always* emit Frontend level
> profile files despite the presence of the -fprofile-generate flag, which
> should make the executable generate IR level profile files. Other
> prerequisites to reproduce this problem including:
> • Using (latest version of) BFD linker
> • Using LLVM’s own LTO linker plugin (i.e. LLVMgold.so)
> • Using LLVM’s compiler-rt as the runtime library
> • On a Linux / BSD platform
> This problem has been confirmed *not* to happen with gold and LLD
> linkers. Impacts on usages on Windows platform remain unknown.
>
> I just cooked a patch for this bug: https://reviews.llvm.org/D83967
> Basically the solution (and the root cause) is similar to this patch
> several years ago: https://reviews.llvm.org/D34797
>
> The intriguing (or annoying) part of this bug is that you’ll never find
> the issue (since it neither crashes nor causes significant performance
> regression) unless you take a look into the generated profile files. So I
> just want to put a heads up here, though I know most of the folks here are
> moving away from using BFD linkers.
>
> Best,
> -Min
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200716/914afd66/attachment.html>

> On Jul 16, 2020, at 11:04 AM, David Major <dmajor at mozilla.com>
wrote:
> 
> This sounds a lot like https://bugs.llvm.org/show_bug.cgi?id=45050
<https://bugs.llvm.org/show_bug.cgi?id=45050>.
Yeah I think that bug is the same as this one. I’ve also left a comment there to
point out my patch review.
> I'm glad to hear you figured out the root cause, thank you!
Best
-Min
> 
> 
> On Thu, Jul 16, 2020 at 1:55 PM Min-Yih Hsu via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hi All,
> 
> Recently we discovered a (hidden) performance bug that happened when you’re
using Full LTO setup + BFD linker to compile first-stage PGO executables that
were expected to generate IR level profile files. That is, compiling an
executable using following commands:
> ```
> clang -flto -fprofile-generate -c sample.c -o sample.c.bc.o
> clang -flto sample.c.bc.o -o sample_exe -fprofile-generate
> ```
> The  resulting sample_exe executable will always emit Frontend level
profile files despite the presence of the -fprofile-generate flag, which should
make the executable generate IR level profile files. Other prerequisites to
reproduce this problem including:
> 	• Using (latest version of) BFD linker
> 	• Using LLVM’s own LTO linker plugin (i.e. LLVMgold.so)
> 	• Using LLVM’s compiler-rt as the runtime library
> 	• On a Linux / BSD platform
> This problem has been confirmed not to happen with gold and LLD linkers.
Impacts on usages on Windows platform remain unknown.
> 
> I just cooked a patch for this bug: https://reviews.llvm.org/D83967
<https://reviews.llvm.org/D83967>
> Basically the solution (and the root cause) is similar to this patch
several years ago: https://reviews.llvm.org/D34797
<https://reviews.llvm.org/D34797>
> 
> The intriguing (or annoying) part of this bug is that you’ll never find the
issue (since it neither crashes nor causes significant performance regression)
unless you take a look into the generated profile files. So I just want to put a
heads up here, though I know most of the folks here are moving away from using
BFD linkers.
> 
> Best,
> -Min
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200716/f8c81503/attachment.html>