ORiordan, Martin via llvm-dev
2017-Jul-18 14:05 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
We are trying to get PGO working for our embedded out-of-tree target, but the utility 'llvm-profdata' does not like the data we are giving it. Because this is not a hosted environment, we have to off-chip the profiling data ourselves, and although the data looks okay, 'llvm-profdata' reports the following error: llvm-profdata show -all-functions -counts -detailed-summary -text -static-func-full-module-prefix ~/Downloads/default.profraw error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) But even for a hosted system, it is not clear to me where 'zlib' gets involved in compressing the data. The start of our data looks like: hexdump -C default.profraw 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 00 |.Rforpl.........| 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 00 |<.......(.......| Which looks valid for Little-Endian, 32-bit and LLVM v4.0. What do we need to do to prepare this raw data from the device into the form that 'llvm-profdata' needs to see? Thanks, MartinO -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170718/9abb0cf3/attachment-0001.html>
You might want to set breakpoint on readPGOFuncNameStrings [1] to see what's going on. [1] http://llvm.org/doxygen/InstrProf_8cpp_source.html 2017-07-18 22:05 GMT+08:00 ORiordan, Martin via llvm-dev <llvm-dev at lists.llvm.org>:> We are trying to get PGO working for our embedded out-of-tree target, but > the utility ‘llvm-profdata’ does not like the data we are giving it. > > > > Because this is not a hosted environment, we have to off-chip the profiling > data ourselves, and although the data looks okay, ‘llvm-profdata’ reports > the following error: > > > > llvm-profdata show -all-functions -counts -detailed-summary -text > -static-func-full-module-prefix ~/Downloads/default.profraw > > error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) > > > > But even for a hosted system, it is not clear to me where ‘zlib’ gets > involved in compressing the data. The start of our data looks like: > > > > hexdump -C default.profraw > > 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 > > 00 |.Rforpl.........| > > 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 > > 00 |<.......(.......| > > > > Which looks valid for Little-Endian, 32-bit and LLVM v4.0. > > > > What do we need to do to prepare this raw data from the device into the form > that ‘llvm-profdata’ needs to see? > > > > Thanks, > > > > MartinO > > > > > > -------------------------------------------------------------- > Intel Research and Development Ireland Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County Kildare > Registered Number: 308263 > > This e-mail and any attachments may contain confidential material for the > sole use of the intended recipient(s). Any review or distribution by others > is strictly prohibited. If you are not the intended recipient, please > contact the sender and delete all copies. > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-- Wei-Ren Chen (陳韋任) Homepage: https://people.cs.nctu.edu.tw/~chenwj
Xinliang David Li via llvm-dev
2017-Jul-18 16:53 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
The __llvm_prf_names section is compressed but your llvm-profdata tool is probably not built with zlib enabled. To disable compression, use option -mllvm -enable-name-compression=false in your instrumentation build. David On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev < llvm-dev at lists.llvm.org> wrote:> We are trying to get PGO working for our embedded out-of-tree target, but > the utility ‘llvm-profdata’ does not like the data we are giving it. > > > > Because this is not a hosted environment, we have to off-chip the > profiling data ourselves, and although the data looks okay, ‘llvm-profdata’ > reports the following error: > > > > *llvm-profdata show -all-functions -counts -detailed-summary -text > -static-func-full-module-prefix ~/Downloads/default.profraw* > > error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) > > > > But even for a hosted system, it is not clear to me where ‘zlib’ gets > involved in compressing the data. The start of our data looks like: > > > > *hexdump -C default.profraw* > > 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 > > 00 |.Rforpl.........| > > 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 > > 00 |<.......(.......| > > > > Which looks valid for Little-Endian, 32-bit and LLVM v4.0. > > > > What do we need to do to prepare this raw data from the device into the > form that ‘llvm-profdata’ needs to see? > > > > Thanks, > > > > MartinO > > > > > > -------------------------------------------------------------- > Intel Research and Development Ireland Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County Kildare > Registered Number: 308263 > > This e-mail and any attachments may contain confidential material for the > sole use of the intended recipient(s). Any review or distribution by others > is strictly prohibited. If you are not the intended recipient, please > contact the sender and delete all copies. > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170718/a749b12e/attachment.html>
Justin Bogner via llvm-dev
2017-Jul-18 18:37 UTC
[llvm-dev] PGO, zlib and 'default.profraw'
Can we improve the error message here? We should be able to check zlib::isAvailable and give an error like "profile uses zlib compression but the profile reader was built without zlib support" or so in this case. Xinliang David Li via llvm-dev <llvm-dev at lists.llvm.org> writes:> The __llvm_prf_names section is compressed but your llvm-profdata tool is > probably not built with zlib enabled. > > To disable compression, use option -mllvm -enable-name-compression=false > in your instrumentation build. > > David > > On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> We are trying to get PGO working for our embedded out-of-tree target, but >> the utility ‘llvm-profdata’ does not like the data we are giving it. >> >> >> >> Because this is not a hosted environment, we have to off-chip the >> profiling data ourselves, and although the data looks okay, ‘llvm-profdata’ >> reports the following error: >> >> >> >> *llvm-profdata show -all-functions -counts -detailed-summary -text >> -static-func-full-module-prefix ~/Downloads/default.profraw* >> >> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib) >> >> >> >> But even for a hosted system, it is not clear to me where ‘zlib’ gets >> involved in compressing the data. The start of our data looks like: >> >> >> >> *hexdump -C default.profraw* >> >> 00000000 81 52 66 6f 72 70 6c ff 04 00 00 00 00 00 00 >> >> 00 |.Rforpl.........| >> >> 00000010 3c 00 00 00 00 00 00 00 28 01 00 00 00 00 00 >> >> 00 |<.......(.......| >> >> >> >> Which looks valid for Little-Endian, 32-bit and LLVM v4.0. >> >> >> >> What do we need to do to prepare this raw data from the device into the >> form that ‘llvm-profdata’ needs to see? >> >> >> >> Thanks, >> >> >> >> MartinO >> >> >> >> >> >> -------------------------------------------------------------- >> Intel Research and Development Ireland Limited >> Registered in Ireland >> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare >> Registered Number: 308263 >> >> This e-mail and any attachments may contain confidential material for the >> sole use of the intended recipient(s). Any review or distribution by others >> is strictly prohibited. If you are not the intended recipient, please >> contact the sender and delete all copies. >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev