Justin Bogner via llvm-dev
2017-Jul-18 18:37 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
Can we improve the error message here? We should be able to check zlib::isAvailable and give an error like "profile uses zlib compression but the profile reader was built without zlib support" or so in this case. Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> writes:> The __llvm_prf_names section is compressed but your llvm-profdata tool is > probably not built with zlib enabled. > > To disable compression, use option -mllvm -enable-name-compression=false > in your instrumentation build. > > David > > On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> We are trying to get PGO working for our embedded out-of-tree target, but >> the utility ‘llvm-profdata’ does not like the data we are giving it. >> >> >> >> Because this is not a hosted environment, we have to off-chip the >> profiling data ourselves, and although the data looks okay, ‘llvm-profdata’ >> reports the following error: >> >> >> >> *llvm-profdata show -all-functions -counts -detailed-summary -text >> -static-func-full-module-prefix ~/Downloads/default.profraw* >> >> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) >> >> >> >> But even for a hosted system, it is not clear to me where ‘zlib’ gets >> involved in compressing the data. The start of our data looks like: >> >> >> >> *hexdump -C default.profraw* >> >> 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 >> >> 00 |.Rforpl.........| >> >> 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 >> >> 00 |<.......(.......| >> >> >> >> Which looks valid for Little-Endian, 32-bit and LLVM v4.0. >> >> >> >> What do we need to do to prepare this raw data from the device into the >> form that ‘llvm-profdata’ needs to see? >> >> >> >> Thanks, >> >> >> >> MartinO >> >> >> >> >> >> -------------------------------------------------------------- >> Intel Research and Development Ireland Limited >> Registered in Ireland >> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare >> Registered Number: 308263 >> >> This e-mail and any attachments may contain confidential material for the >> sole use of the intended recipient(s). Any review or distribution by others >> is strictly prohibited. If you are not the intended recipient, please >> contact the sender and delete all copies. >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Xinliang David Li via llvm-dev
2017-Jul-18 18:39 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
I think that is a good idea. David On Tue, Jul 18, 2017 at 11:37 AM, Justin Bogner <mail at justinbogner.com> wrote:> Can we improve the error message here? We should be able to check > zlib::isAvailable and give an error like "profile uses zlib compression > but the profile reader was built without zlib support" or so in this > case. > > Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> writes: > > The __llvm_prf_names section is compressed but your llvm-profdata tool is > > probably not built with zlib enabled. > > > > To disable compression, use option -mllvm -enable-name-compression=false > > in your instrumentation build. > > > > David > > > > On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev < > > llvm-dev at lists.llvm.org> wrote: > > > >> We are trying to get PGO working for our embedded out-of-tree target, > but > >> the utility ‘llvm-profdata’ does not like the data we are giving it. > >> > >> > >> > >> Because this is not a hosted environment, we have to off-chip the > >> profiling data ourselves, and although the data looks okay, > ‘llvm-profdata’ > >> reports the following error: > >> > >> > >> > >> *llvm-profdata show -all-functions -counts -detailed-summary -text > >> -static-func-full-module-prefix ~/Downloads/default.profraw* > >> > >> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) > >> > >> > >> > >> But even for a hosted system, it is not clear to me where ‘zlib’ gets > >> involved in compressing the data. The start of our data looks like: > >> > >> > >> > >> *hexdump -C default.profraw* > >> > >> 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 > >> > >> 00 |.Rforpl.........| > >> > >> 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 > >> > >> 00 |<.......(.......| > >> > >> > >> > >> Which looks valid for Little-Endian, 32-bit and LLVM v4.0. > >> > >> > >> > >> What do we need to do to prepare this raw data from the device into the > >> form that ‘llvm-profdata’ needs to see? > >> > >> > >> > >> Thanks, > >> > >> > >> > >> MartinO > >> > >> > >> > >> > >> > >> -------------------------------------------------------------- > >> Intel Research and Development Ireland Limited > >> Registered in Ireland > >> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare > >> Registered Number: 308263 > >> > >> This e-mail and any attachments may contain confidential material for > the > >> sole use of the intended recipient(s). Any review or distribution by > others > >> is strictly prohibited. If you are not the intended recipient, please > >> contact the sender and delete all copies. > >> > >> > >> _______________________________________________ > >> LLVM Developers mailing list > >> llvm-dev at lists.llvm.org > >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >> > >> > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170718/6270e5a4/attachment.html>
Martin J. O'Riordan via llvm-dev
2017-Jul-18 20:02 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
Thanks for the feedback. I am off for a short well deserved vacation :), but
when I get back I will put these suggestions to good use.
All the best,
MartinO
From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Xinliang
David Li via llvm-dev
Sent: 18 July 2017 19:39
To: Justin Bogner <mail at justinbogner.com>
Cc: Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] PGO, zlib and 'default.profraw'
I think that is a good idea.
David
On Tue, Jul 18, 2017 at 11:37 AM, Justin Bogner <mail at justinbogner.com
<mailto:mail at justinbogner.com> > wrote:
Can we improve the error message here? We should be able to check
zlib::isAvailable and give an error like "profile uses zlib compression
but the profile reader was built without zlib support" or so in this
case.
Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org> > writes:> The __llvm_prf_names section is compressed but your llvm-profdata tool is
> probably not built with zlib enabled.
>
> To disable compression, use option -mllvm -enable-name-compression=false
> in your instrumentation build.
>
> David
>
> On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev <
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >
wrote:
>
>> We are trying to get PGO working for our embedded out-of-tree target,
but
>> the utility ‘llvm-profdata’ does not like the data we are giving it.
>>
>>
>>
>> Because this is not a hosted environment, we have to off-chip the
>> profiling data ourselves, and although the data looks okay,
‘llvm-profdata’
>> reports the following error:
>>
>>
>>
>> *llvm-profdata show -all-functions -counts -detailed-summary -text
>> -static-func-full-module-prefix ~/Downloads/default.profraw*
>>
>> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib)
>>
>>
>>
>> But even for a hosted system, it is not clear to me where ‘zlib’ gets
>> involved in compressing the data. The start of our data looks like:
>>
>>
>>
>> *hexdump -C default.profraw*
>>
>> 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00
>>
>> 00 |.Rforpl.........|
>>
>> 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00
>>
>> 00 |<.......(.......|
>>
>>
>>
>> Which looks valid for Little-Endian, 32-bit and LLVM v4.0.
>>
>>
>>
>> What do we need to do to prepare this raw data from the device into the
>> form that ‘llvm-profdata’ needs to see?
>>
>>
>>
>> Thanks,
>>
>>
>>
>> MartinO
>>
>>
>>
>>
>>
>> --------------------------------------------------------------
>> Intel Research and Development Ireland Limited
>> Registered in Ireland
>> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
>> Registered Number: 308263
>>
>> This e-mail and any attachments may contain confidential material for
the
>> sole use of the intended recipient(s). Any review or distribution by
others
>> is strictly prohibited. If you are not the intended recipient, please
>> contact the sender and delete all copies.
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170718/d183b8d9/attachment.html>
Martin J. O'Riordan via llvm-dev
2017-Jul-18 20:04 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
How can I build the profile reader with ZLIB support enabled? I configure and build all the tools together with 'clang', so I would have expected that all or none support ZLIB, or is this a cross-compiler configuration specific issue? Thanks, MartinO -----Original Message----- From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Justin Bogner via llvm-dev Sent: 18 July 2017 19:37 To: Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> Subject: Re: [llvm-dev] PGO, zlib and 'default.profraw' Can we improve the error message here? We should be able to check zlib::isAvailable and give an error like "profile uses zlib compression but the profile reader was built without zlib support" or so in this case. Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> writes:> The __llvm_prf_names section is compressed but your llvm-profdata tool > is probably not built with zlib enabled. > > To disable compression, use option -mllvm > -enable-name-compression=false in your instrumentation build. > > David > > On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> We are trying to get PGO working for our embedded out-of-tree target, >> but the utility ‘llvm-profdata’ does not like the data we are giving it. >> >> >> >> Because this is not a hosted environment, we have to off-chip the >> profiling data ourselves, and although the data looks okay, ‘llvm-profdata’ >> reports the following error: >> >> >> >> *llvm-profdata show -all-functions -counts -detailed-summary -text >> -static-func-full-module-prefix ~/Downloads/default.profraw* >> >> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) >> >> >> >> But even for a hosted system, it is not clear to me where ‘zlib’ gets >> involved in compressing the data. The start of our data looks like: >> >> >> >> *hexdump -C default.profraw* >> >> 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 >> >> 00 |.Rforpl.........| >> >> 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 >> >> 00 |<.......(.......| >> >> >> >> Which looks valid for Little-Endian, 32-bit and LLVM v4.0. >> >> >> >> What do we need to do to prepare this raw data from the device into >> the form that ‘llvm-profdata’ needs to see? >> >> >> >> Thanks, >> >> >> >> MartinO >> >> >> >> >> >> -------------------------------------------------------------- >> Intel Research and Development Ireland Limited Registered in Ireland >> Registered Office: Collinstown Industrial Park, Leixlip, County >> Kildare Registered Number: 308263 >> >> This e-mail and any attachments may contain confidential material for >> the sole use of the intended recipient(s). Any review or distribution >> by others is strictly prohibited. If you are not the intended >> recipient, please contact the sender and delete all copies. >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev_______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Xinliang David Li via llvm-dev
2017-Jul-18 21:40 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
set LLVM_ENABLE_ZLIB=ON with cmake invocation. zlib should be installed and zlib.h header file needs to be in the header search path. Is your llvm-profdata tool built together with clang? David On Tue, Jul 18, 2017 at 1:04 PM, Martin J. O'Riordan via llvm-dev < llvm-dev at lists.llvm.org> wrote:> How can I build the profile reader with ZLIB support enabled? I configure > and build all the tools together with 'clang', so I would have expected > that all or none support ZLIB, or is this a cross-compiler configuration > specific issue? > > Thanks, > > MartinO > > -----Original Message----- > From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of > Justin Bogner via llvm-dev > Sent: 18 July 2017 19:37 > To: Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> > Subject: Re: [llvm-dev] PGO, zlib and 'default.profraw' > > Can we improve the error message here? We should be able to check > zlib::isAvailable and give an error like "profile uses zlib compression but > the profile reader was built without zlib support" or so in this case. > > Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> writes: > > The __llvm_prf_names section is compressed but your llvm-profdata tool > > is probably not built with zlib enabled. > > > > To disable compression, use option -mllvm > > -enable-name-compression=false in your instrumentation build. > > > > David > > > > On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev < > > llvm-dev at lists.llvm.org> wrote: > > > >> We are trying to get PGO working for our embedded out-of-tree target, > >> but the utility ‘llvm-profdata’ does not like the data we are giving it. > >> > >> > >> > >> Because this is not a hosted environment, we have to off-chip the > >> profiling data ourselves, and although the data looks okay, > ‘llvm-profdata’ > >> reports the following error: > >> > >> > >> > >> *llvm-profdata show -all-functions -counts -detailed-summary -text > >> -static-func-full-module-prefix ~/Downloads/default.profraw* > >> > >> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) > >> > >> > >> > >> But even for a hosted system, it is not clear to me where ‘zlib’ gets > >> involved in compressing the data. The start of our data looks like: > >> > >> > >> > >> *hexdump -C default.profraw* > >> > >> 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 > >> > >> 00 |.Rforpl.........| > >> > >> 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 > >> > >> 00 |<.......(.......| > >> > >> > >> > >> Which looks valid for Little-Endian, 32-bit and LLVM v4.0. > >> > >> > >> > >> What do we need to do to prepare this raw data from the device into > >> the form that ‘llvm-profdata’ needs to see? > >> > >> > >> > >> Thanks, > >> > >> > >> > >> MartinO > >> > >> > >> > >> > >> > >> -------------------------------------------------------------- > >> Intel Research and Development Ireland Limited Registered in Ireland > >> Registered Office: Collinstown Industrial Park, Leixlip, County > >> Kildare Registered Number: 308263 > >> > >> This e-mail and any attachments may contain confidential material for > >> the sole use of the intended recipient(s). Any review or distribution > >> by others is strictly prohibited. If you are not the intended > >> recipient, please contact the sender and delete all copies. > >> > >> > >> _______________________________________________ > >> LLVM Developers mailing list > >> llvm-dev at lists.llvm.org > >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >> > >> > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170718/3fb8e0ce/attachment.html>