llvm dev - Dec 2015 - InstrProf backward compatibility

Hi all,
  I am working on adding PGO to LDC (LLVM D Compiler). The current
implementation
1) uses LLVM's InstrProf pass to generate an instrumented executable
2) links to compiler-rt/lib/profile for the runtime functionality to write
a raw profile data file
3) uses llvm-profdata to merge profile data and convert from profraw to
profdata format
4) uses llvm::IndexedInstrProfReader to read-in the profdata file for the
2nd PGO compile pass

This all works well with LLVM 3.7. While testing this with LLVM trunk, I
run into a version compatibility problem between different versions of
LLVM: I made a local copy of the compiler-rt/lib/profile release_37 version
(with a few fixes for MSVC) and so steps 1,2, and 4 use LLVM trunk, but
step 2 uses an older version of compiler-rt/lib/profile. Apparently,
trunk's llvm-profdata cannot read raw profile data files generated by
compiler-rt 3.7. Is this intended? Is llvm-profdata's output also
incompatible between different versions of LLVM (InstrProfWriter/Reader are
not meant to be compatible across LLVM versions) ?

LDC is currently compatible with LLVM 3.5 up to trunk. To support PGO for
multiple LLVM versions (>= 3.7), I think that we will need to have a copy
of compiler-rt/lib/profile for each version of LLVM >= 3.7. We also need to
have a copy of llvm-profdata for each version of LLVM >= 3.7. We already do
this with the FileCheck utility and will not be a big problem.

Does that make sense? What do you advise?

Thanks very much for your comments,
  Johan
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151218/05e0ac5a/attachment.html>

Hi Johan,
> On Dec 18, 2015, at 5:37 AM, Johan Engelen via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi all,
>   I am working on adding PGO to LDC (LLVM D Compiler). The current
implementation
> 1) uses LLVM's InstrProf pass to generate an instrumented executable
> 2) links to compiler-rt/lib/profile for the runtime functionality to write
a raw profile data file
> 3) uses llvm-profdata to merge profile data and convert from profraw to
profdata format
> 4) uses llvm::IndexedInstrProfReader to read-in the profdata file for the
2nd PGO compile pass
> 
> This all works well with LLVM 3.7. While testing this with LLVM trunk, I
run into a version compatibility problem between different versions of LLVM: I
made a local copy of the compiler-rt/lib/profile release_37 version (with a few
fixes for MSVC) and so steps 1,2, and 4 use LLVM trunk, but step 2 uses an older
version of compiler-rt/lib/profile. Apparently, trunk's llvm-profdata cannot
read raw profile data files generated by compiler-rt 3.7. Is this intended?
Yes, as far as I'm aware, the raw profile format is meant to buy llvm
flexibility. There are backwards compatibility guarantees for the indexed
(*.profdata) format. I'm not sure if these guarantees have been written down
yet -- if they haven't I'll put a document together.

The profraw/profdata split is analogous to the textual IR / bitcode split.

> Is llvm-profdata's output also incompatible between different versions
of LLVM (InstrProfWriter/Reader are not meant to be compatible across LLVM
versions) ?
The indexed profdata reader for version X+1 of the format should be able to read
versions <= X. There are no forward-compatibility promises: the reader for
version X may not be able to read version X+k files.

> LDC is currently compatible with LLVM 3.5 up to trunk. To support PGO for
multiple LLVM versions (>= 3.7), I think that we will need to have a copy of
compiler-rt/lib/profile for each version of LLVM >= 3.7. We also need to have
a copy of llvm-profdata for each version of LLVM >= 3.7. We already do this
with the FileCheck utility and will not be a big problem.
You should only ever need the latest compiler-rt you can use (i.e, one
compatible with your version of llvm). That should work if you only rely on
backwards compatibility for *.profdata files.

Does this work for your purposes? Is there anything we can do to make life
easier for downstream projects?


best,
vedant

On Fri, Dec 18, 2015 at 5:37 AM, Johan Engelen <jbc.engelen at gmail.com>
wrote:
> Hi all,
>   I am working on adding PGO to LDC (LLVM D Compiler). The current
> implementation
> 1) uses LLVM's InstrProf pass to generate an instrumented executable
> 2) links to compiler-rt/lib/profile for the runtime functionality to write
a
> raw profile data file
> 3) uses llvm-profdata to merge profile data and convert from profraw to
> profdata format
> 4) uses llvm::IndexedInstrProfReader to read-in the profdata file for the
> 2nd PGO compile pass
>
> This all works well with LLVM 3.7. While testing this with LLVM trunk, I
run
> into a version compatibility problem between different versions of LLVM: I
> made a local copy of the compiler-rt/lib/profile release_37 version (with a
> few fixes for MSVC) and so steps 1,2, and 4 use LLVM trunk, but step 2 uses
> an older version of compiler-rt/lib/profile. Apparently, trunk's
Runtime and InstrProf pass have to be in sync. You can not use 3.7
version of runtime with trunk version of 1).

Perhaps you want to explain more about your use cases? If you want to
use 3.7 compiler/runtime to produce profile to be used by trunk
compiler -- that is doable.

1) instrument the code with 3.7 compiler and link with 3.7 runtime
2) run and dump the profile
3) use 3.7 llvm-profdata tool to do a merge and generate indexed format profile
4) use that profile with the trunk compiler

However, the other direction does not work -- you can not expect older
version of compiler to consume indexed format profile of the new
compiler.

> llvm-profdata cannot read raw profile data files generated by compiler-rt
> 3.7. Is this intended?
yes. New version of raw profile reader does not understand old version
of raw profile data. You need to use older version of llvm-profdata to
produce old version of indexed format, and then use new version of
llvm-profdata to convert the indexed format into new version. Here is
flow:

1) llvm-profdata_3.7 merge -o my_profdata_old.profdata my_profdata_old.profraw

At this point, my_profdata_old.profdata *can* be consumed by trunk
version of compiler already, but if you want to convert it to new
version of indexed format, do this:

2) llvm-profdata_trunk merge -o my_profdata_new.profdata
my_profdata_old.profdata

Now my_profdata_new.profdata is in shiny new version (indexed).


 Is llvm-profdata's output also incompatible between
> different versions of LLVM (InstrProfWriter/Reader are not meant to be
> compatible across LLVM versions) ?
The merge output format (indexed format) is backward compatible as
explained by Vedant.
>
> LDC is currently compatible with LLVM 3.5 up to trunk. To support PGO for
> multiple LLVM versions (>= 3.7), I think that we will need to have a
copy of
> compiler-rt/lib/profile for each version of LLVM >= 3.7. We also need to
> have a copy of llvm-profdata for each version of LLVM >= 3.7. We already
do
> this with the FileCheck utility and will not be a big problem.
>
yes -- see above. Remember compiler instrumentation and supporting
runtime must be in sync.

thanks,

David

> Does that make sense? What do you advise?
>
> Thanks very much for your comments,
>   Johan
>

Thanks for the clear answers.

At the moment, my only request would be to also make sure that profile lib
works with MSVC (3.7 needed small patching, trunk is fine now).

Thanks!
  Johan


On Fri, Dec 18, 2015 at 7:49 PM, Xinliang David Li <davidxl at google.com>
wrote:
> On Fri, Dec 18, 2015 at 5:37 AM, Johan Engelen <jbc.engelen at
gmail.com>
> wrote:
> > Hi all,
> >   I am working on adding PGO to LDC (LLVM D Compiler). The current
> > implementation
> > 1) uses LLVM's InstrProf pass to generate an instrumented
executable
> > 2) links to compiler-rt/lib/profile for the runtime functionality to
> write a
> > raw profile data file
> > 3) uses llvm-profdata to merge profile data and convert from profraw
to
> > profdata format
> > 4) uses llvm::IndexedInstrProfReader to read-in the profdata file for
the
> > 2nd PGO compile pass
> >
> > This all works well with LLVM 3.7. While testing this with LLVM trunk,
I
> run
> > into a version compatibility problem between different versions of
LLVM:
> I
> > made a local copy of the compiler-rt/lib/profile release_37 version
> (with a
> > few fixes for MSVC) and so steps 1,2, and 4 use LLVM trunk, but step 2
> uses
> > an older version of compiler-rt/lib/profile. Apparently, trunk's
>
> Runtime and InstrProf pass have to be in sync. You can not use 3.7
> version of runtime with trunk version of 1).
>
> Perhaps you want to explain more about your use cases? If you want to
> use 3.7 compiler/runtime to produce profile to be used by trunk
> compiler -- that is doable.
>
> 1) instrument the code with 3.7 compiler and link with 3.7 runtime
> 2) run and dump the profile
> 3) use 3.7 llvm-profdata tool to do a merge and generate indexed format
> profile
> 4) use that profile with the trunk compiler
>
> However, the other direction does not work -- you can not expect older
> version of compiler to consume indexed format profile of the new
> compiler.
>
>
> > llvm-profdata cannot read raw profile data files generated by
compiler-rt
> > 3.7. Is this intended?
>
> yes. New version of raw profile reader does not understand old version
> of raw profile data. You need to use older version of llvm-profdata to
> produce old version of indexed format, and then use new version of
> llvm-profdata to convert the indexed format into new version. Here is
> flow:
>
> 1) llvm-profdata_3.7 merge -o my_profdata_old.profdata
> my_profdata_old.profraw
>
> At this point, my_profdata_old.profdata *can* be consumed by trunk
> version of compiler already, but if you want to convert it to new
> version of indexed format, do this:
>
> 2) llvm-profdata_trunk merge -o my_profdata_new.profdata
> my_profdata_old.profdata
>
> Now my_profdata_new.profdata is in shiny new version (indexed).
>
>
>  Is llvm-profdata's output also incompatible between
> > different versions of LLVM (InstrProfWriter/Reader are not meant to be
> > compatible across LLVM versions) ?
>
> The merge output format (indexed format) is backward compatible as
> explained by Vedant.
>
> >
> > LDC is currently compatible with LLVM 3.5 up to trunk. To support PGO
for
> > multiple LLVM versions (>= 3.7), I think that we will need to have
a
> copy of
> > compiler-rt/lib/profile for each version of LLVM >= 3.7. We also
need to
> > have a copy of llvm-profdata for each version of LLVM >= 3.7. We
already
> do
> > this with the FileCheck utility and will not be a big problem.
> >
>
> yes -- see above. Remember compiler instrumentation and supporting
> runtime must be in sync.
>
> thanks,
>
> David
>
>
> > Does that make sense? What do you advise?
> >
> > Thanks very much for your comments,
> >   Johan
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151219/53d1dc72/attachment.html>