llvm dev - Sep 2019 - Struggling with a PGO build of clang -- llvm-profdata was built without zlib support?

Hi!

I'm trying to build a fast Clang for myself to use for debug builds on
Clang itself, but I've been struggling for a very long time on it. Could
you please help?

I've been following this guide: https://llvm.org/docs/HowToBuildWithPGO.html

I've quickly learned that its outdated, because the script it talks about
doesn't work with the monorepo layout at all, but in any case, it does
describe how to do the entire process by hand, so I went for that. Here are
my steps:

* Built a release version of LLVM/Clang/LLD/compiler-rt on version 9.0.0-rc3

* Built an instrumented clang with the following config:
cmake -G Ninja ../llvm \
  -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' \
  -DLLVM_TARGETS_TO_BUILD=X86 \
  -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
  -DCMAKE_CXX_COMPILER=/path/to/release_build/bin/clang++ \
  -DCMAKE_C_COMPILER=/path/to/release_build/bin/clang \
  -DLLVM_USE_LINKER=/path/to/release_build/bin/ld.lld \
  -DLLVM_CCACHE_BUILD=ON \
  -DLLVM_BUILD_INSTRUMENTED=IR \
  -DLLVM_BUILD_RUNTIME=No \
  -DLLVM_ENABLE_ZLIB=1 \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_LTO=Thin \
  -DCMAKE_RANLIB=/path/to/release_build/bin/llvm-ranlib \
  -DCMAKE_AR=/path/to/release_build/bin/llvm-ar

I tried this without specifying zlib being enabled, without thinlto,
everything I could come up with really.

* Built a debug Clang with the following configuration:
cmake -G Ninja ../llvm \
  -DCMAKE_BUILD_TYPE=Debug \
  -DLLVM_ENABLE_PROJECTS='clang' \
  -DBUILD_SHARED_LIBS=ON \
  -DLLVM_TARGETS_TO_BUILD=X86 \
  -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
  -DCMAKE_CXX_COMPILER=/path/to/instrumented_build/bin/clang++ \
  -DCMAKE_C_COMPILER=/path/to/instrumented_build/bin/clang \
  -DLLVM_USE_LINKER=/path/to/release_build/bin/ld.lld  \
  -DLLVM_CCACHE_BUILD=ON

* Tried to use /path/to/instrumented_build/llvm-profdata on the profraw
files found in  /path/to/instrumented_build/profiles/.

I got the following error message:
error: profiles/default_15822678447050577402_0.profraw: Profile uses zlib
compression but the profile reader was built without zlib support

Even when I explicitly specified zlib to be enabled, and double check
whether zlib.h was found I got this message. I've found this [1] thread
mentioning a similar issue, but I'm unfortunately not familiar enough with
LLVM itself to use the discussed tips with any success. Is there something
obvious I'm missing?

Thanks in advance,
Kristóf

[1] http://lists.llvm.org/pipermail/llvm-dev/2017-July/115588.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190903/583ea6ee/attachment.html>

I /guess/ you actually want /path/to/release_build/llvm-profdata because
the profiles are generated from binaries compiled with the release build,
so it's the release build that matters for all things profiling related, I
would think, in your situation.


On Tue, Sep 3, 2019 at 10:06 AM Kristóf Umann via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi!
>
> I'm trying to build a fast Clang for myself to use for debug builds on
> Clang itself, but I've been struggling for a very long time on it.
Could
> you please help?
>
> I've been following this guide:
> https://llvm.org/docs/HowToBuildWithPGO.html
>
> I've quickly learned that its outdated, because the script it talks
about
> doesn't work with the monorepo layout at all, but in any case, it does
> describe how to do the entire process by hand, so I went for that. Here are
> my steps:
>
> * Built a release version of LLVM/Clang/LLD/compiler-rt on version
> 9.0.0-rc3
>
> * Built an instrumented clang with the following config:
> cmake -G Ninja ../llvm \
>   -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' \
>   -DLLVM_TARGETS_TO_BUILD=X86 \
>   -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
>   -DCMAKE_CXX_COMPILER=/path/to/release_build/bin/clang++ \
>   -DCMAKE_C_COMPILER=/path/to/release_build/bin/clang \
>   -DLLVM_USE_LINKER=/path/to/release_build/bin/ld.lld \
>   -DLLVM_CCACHE_BUILD=ON \
>   -DLLVM_BUILD_INSTRUMENTED=IR \
>   -DLLVM_BUILD_RUNTIME=No \
>   -DLLVM_ENABLE_ZLIB=1 \
>   -DCMAKE_BUILD_TYPE=Release \
>   -DLLVM_ENABLE_LTO=Thin \
>   -DCMAKE_RANLIB=/path/to/release_build/bin/llvm-ranlib \
>   -DCMAKE_AR=/path/to/release_build/bin/llvm-ar
>
> I tried this without specifying zlib being enabled, without thinlto,
> everything I could come up with really.
>
> * Built a debug Clang with the following configuration:
> cmake -G Ninja ../llvm \
>   -DCMAKE_BUILD_TYPE=Debug \
>   -DLLVM_ENABLE_PROJECTS='clang' \
>   -DBUILD_SHARED_LIBS=ON \
>   -DLLVM_TARGETS_TO_BUILD=X86 \
>   -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
>   -DCMAKE_CXX_COMPILER=/path/to/instrumented_build/bin/clang++ \
>   -DCMAKE_C_COMPILER=/path/to/instrumented_build/bin/clang \
>   -DLLVM_USE_LINKER=/path/to/release_build/bin/ld.lld  \
>   -DLLVM_CCACHE_BUILD=ON
>
> * Tried to use /path/to/instrumented_build/llvm-profdata on the profraw
> files found in  /path/to/instrumented_build/profiles/.
>
> I got the following error message:
> error: profiles/default_15822678447050577402_0.profraw: Profile uses zlib
> compression but the profile reader was built without zlib support
>
> Even when I explicitly specified zlib to be enabled, and double check
> whether zlib.h was found I got this message. I've found this [1] thread
> mentioning a similar issue, but I'm unfortunately not familiar enough
with
> LLVM itself to use the discussed tips with any success. Is there something
> obvious I'm missing?
>
> Thanks in advance,
> Kristóf
>
> [1] http://lists.llvm.org/pipermail/llvm-dev/2017-July/115588.html
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190903/e0195b74/attachment.html>

Yes, that was it! Now that I took a closer look, the guide also states that
I should use the stage2 build. Silly me.

Thanks!

On Tue, 3 Sep 2019 at 19:31, David Blaikie <dblaikie at gmail.com> wrote:
> I /guess/ you actually want /path/to/release_build/llvm-profdata because
> the profiles are generated from binaries compiled with the release build,
> so it's the release build that matters for all things profiling
related, I
> would think, in your situation.
>
>
> On Tue, Sep 3, 2019 at 10:06 AM Kristóf Umann via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi!
>>
>> I'm trying to build a fast Clang for myself to use for debug builds
on
>> Clang itself, but I've been struggling for a very long time on it.
Could
>> you please help?
>>
>> I've been following this guide:
>> https://llvm.org/docs/HowToBuildWithPGO.html
>>
>> I've quickly learned that its outdated, because the script it talks
about
>> doesn't work with the monorepo layout at all, but in any case, it
does
>> describe how to do the entire process by hand, so I went for that. Here
are
>> my steps:
>>
>> * Built a release version of LLVM/Clang/LLD/compiler-rt on version
>> 9.0.0-rc3
>>
>> * Built an instrumented clang with the following config:
>> cmake -G Ninja ../llvm \
>>   -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' \
>>   -DLLVM_TARGETS_TO_BUILD=X86 \
>>   -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
>>   -DCMAKE_CXX_COMPILER=/path/to/release_build/bin/clang++ \
>>   -DCMAKE_C_COMPILER=/path/to/release_build/bin/clang \
>>   -DLLVM_USE_LINKER=/path/to/release_build/bin/ld.lld \
>>   -DLLVM_CCACHE_BUILD=ON \
>>   -DLLVM_BUILD_INSTRUMENTED=IR \
>>   -DLLVM_BUILD_RUNTIME=No \
>>   -DLLVM_ENABLE_ZLIB=1 \
>>   -DCMAKE_BUILD_TYPE=Release \
>>   -DLLVM_ENABLE_LTO=Thin \
>>   -DCMAKE_RANLIB=/path/to/release_build/bin/llvm-ranlib \
>>   -DCMAKE_AR=/path/to/release_build/bin/llvm-ar
>>
>> I tried this without specifying zlib being enabled, without thinlto,
>> everything I could come up with really.
>>
>> * Built a debug Clang with the following configuration:
>> cmake -G Ninja ../llvm \
>>   -DCMAKE_BUILD_TYPE=Debug \
>>   -DLLVM_ENABLE_PROJECTS='clang' \
>>   -DBUILD_SHARED_LIBS=ON \
>>   -DLLVM_TARGETS_TO_BUILD=X86 \
>>   -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
>>   -DCMAKE_CXX_COMPILER=/path/to/instrumented_build/bin/clang++ \
>>   -DCMAKE_C_COMPILER=/path/to/instrumented_build/bin/clang \
>>   -DLLVM_USE_LINKER=/path/to/release_build/bin/ld.lld  \
>>   -DLLVM_CCACHE_BUILD=ON
>>
>> * Tried to use /path/to/instrumented_build/llvm-profdata on the profraw
>> files found in  /path/to/instrumented_build/profiles/.
>>
>> I got the following error message:
>> error: profiles/default_15822678447050577402_0.profraw: Profile uses
zlib
>> compression but the profile reader was built without zlib support
>>
>> Even when I explicitly specified zlib to be enabled, and double check
>> whether zlib.h was found I got this message. I've found this [1]
thread
>> mentioning a similar issue, but I'm unfortunately not familiar
enough with
>> LLVM itself to use the discussed tips with any success. Is there
something
>> obvious I'm missing?
>>
>> Thanks in advance,
>> Kristóf
>>
>> [1] http://lists.llvm.org/pipermail/llvm-dev/2017-July/115588.html
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190903/e3b2c3ea/attachment.html>