llvm dev - Jul 2017 - PGO, zlib and 'default.profraw'

We are trying to get PGO working for our embedded out-of-tree target, but the
utility 'llvm-profdata' does not like the data we are giving it.

Because this is not a hosted environment, we have to off-chip the profiling data
ourselves, and although the data looks okay, 'llvm-profdata' reports the
following error:


llvm-profdata show -all-functions -counts -detailed-summary -text
-static-func-full-module-prefix ~/Downloads/default.profraw

error: ~/Downloads/default.profraw: Failed to uncompress data (zlib)

But even for a hosted system, it is not clear to me where 'zlib' gets
involved in compressing the data.  The start of our data looks like:


hexdump -C default.profraw

00000000  81 52 66 6f 72 70 6c ff  04 00 00 00 00 00 00

00  |.Rforpl.........|

00000010  3c 00 00 00 00 00 00 00  28 01 00 00 00 00 00

00  |<.......(.......|

Which looks valid for Little-Endian, 32-bit and LLVM v4.0.

What do we need to do to prepare this raw data from the device into the form
that 'llvm-profdata' needs to see?

Thanks,

            MartinO


--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170718/9abb0cf3/attachment-0001.html>

You might want to set breakpoint on readPGOFuncNameStrings [1] to see
what's going on.

[1] http://llvm.org/doxygen/InstrProf_8cpp_source.html

2017-07-18 22:05 GMT+08:00 ORiordan, Martin via llvm-dev
<llvm-dev at lists.llvm.org>:
> We are trying to get PGO working for our embedded out-of-tree target, but
> the utility ‘llvm-profdata’ does not like the data we are giving it.
>
>
>
> Because this is not a hosted environment, we have to off-chip the profiling
> data ourselves, and although the data looks okay, ‘llvm-profdata’ reports
> the following error:
>
>
>
> llvm-profdata show -all-functions -counts -detailed-summary -text
> -static-func-full-module-prefix ~/Downloads/default.profraw
>
> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib)
>
>
>
> But even for a hosted system, it is not clear to me where ‘zlib’ gets
> involved in compressing the data.  The start of our data looks like:
>
>
>
> hexdump -C default.profraw
>
> 00000000  81 52 66 6f 72 70 6c ff  04 00 00 00 00 00 00
>
> 00  |.Rforpl.........|
>
> 00000010  3c 00 00 00 00 00 00 00  28 01 00 00 00 00 00
>
> 00  |<.......(.......|
>
>
>
> Which looks valid for Little-Endian, 32-bit and LLVM v4.0.
>
>
>
> What do we need to do to prepare this raw data from the device into the
form
> that ‘llvm-profdata’ needs to see?
>
>
>
> Thanks,
>
>
>
>             MartinO
>
>
>
>
>
> --------------------------------------------------------------
> Intel Research and Development Ireland Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please
> contact the sender and delete all copies.
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>


-- 
Wei-Ren Chen (陳韋任)
Homepage: https://people.cs.nctu.edu.tw/~chenwj

The __llvm_prf_names section is compressed but your llvm-profdata tool is
probably not built with zlib enabled.

To disable compression, use option -mllvm -enable-name-compression=false
 in your instrumentation build.

David

On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> We are trying to get PGO working for our embedded out-of-tree target, but
> the utility ‘llvm-profdata’ does not like the data we are giving it.
>
>
>
> Because this is not a hosted environment, we have to off-chip the
> profiling data ourselves, and although the data looks okay, ‘llvm-profdata’
> reports the following error:
>
>
>
> *llvm-profdata show -all-functions -counts -detailed-summary -text
> -static-func-full-module-prefix ~/Downloads/default.profraw*
>
> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib)
>
>
>
> But even for a hosted system, it is not clear to me where ‘zlib’ gets
> involved in compressing the data.  The start of our data looks like:
>
>
>
> *hexdump -C default.profraw*
>
> 00000000  81 52 66 6f 72 70 6c ff  04 00 00 00 00 00 00
>
> 00  |.Rforpl.........|
>
> 00000010  3c 00 00 00 00 00 00 00  28 01 00 00 00 00 00
>
> 00  |<.......(.......|
>
>
>
> Which looks valid for Little-Endian, 32-bit and LLVM v4.0.
>
>
>
> What do we need to do to prepare this raw data from the device into the
> form that ‘llvm-profdata’ needs to see?
>
>
>
> Thanks,
>
>
>
>             MartinO
>
>
>
>
>
> --------------------------------------------------------------
> Intel Research and Development Ireland Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please
> contact the sender and delete all copies.
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170718/a749b12e/attachment.html>

Can we improve the error message here? We should be able to check
zlib::isAvailable and give an error like "profile uses zlib compression
but the profile reader was built without zlib support" or so in this
case.

Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org>
writes:
> The __llvm_prf_names section is compressed but your llvm-profdata tool is
> probably not built with zlib enabled.
>
> To disable compression, use option -mllvm -enable-name-compression=false
>  in your instrumentation build.
>
> David
>
> On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> We are trying to get PGO working for our embedded out-of-tree target,
but
>> the utility ‘llvm-profdata’ does not like the data we are giving it.
>>
>>
>>
>> Because this is not a hosted environment, we have to off-chip the
>> profiling data ourselves, and although the data looks okay,
‘llvm-profdata’
>> reports the following error:
>>
>>
>>
>> *llvm-profdata show -all-functions -counts -detailed-summary -text
>> -static-func-full-module-prefix ~/Downloads/default.profraw*
>>
>> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib)
>>
>>
>>
>> But even for a hosted system, it is not clear to me where ‘zlib’ gets
>> involved in compressing the data.  The start of our data looks like:
>>
>>
>>
>> *hexdump -C default.profraw*
>>
>> 00000000  81 52 66 6f 72 70 6c ff  04 00 00 00 00 00 00
>>
>> 00  |.Rforpl.........|
>>
>> 00000010  3c 00 00 00 00 00 00 00  28 01 00 00 00 00 00
>>
>> 00  |<.......(.......|
>>
>>
>>
>> Which looks valid for Little-Endian, 32-bit and LLVM v4.0.
>>
>>
>>
>> What do we need to do to prepare this raw data from the device into the
>> form that ‘llvm-profdata’ needs to see?
>>
>>
>>
>> Thanks,
>>
>>
>>
>>             MartinO
>>
>>
>>
>>
>>
>> --------------------------------------------------------------
>> Intel Research and Development Ireland Limited
>> Registered in Ireland
>> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
>> Registered Number: 308263
>>
>> This e-mail and any attachments may contain confidential material for
the
>> sole use of the intended recipient(s). Any review or distribution by
others
>> is strictly prohibited. If you are not the intended recipient, please
>> contact the sender and delete all copies.
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev