llvm dev - May 2012 - [LLVMdev] LTO for smaller memory footprint for Clang

Hi all,

I was trying to use LTO facility of LLVM to reduce the footprint of
Clang itself. I build the ld-gold  and LLVMgold.so as described at [1]
and then set the environment as described too. However, had to add the
path for plugin manually as Clang was not able pass it to ld
automatically. Following is the setting I used before starting to
build (small foot Clang.

CXX=clang++ -flto
-Wl,--plugin=/home/wer/llvm-3.0/build_config_gold_release/Release/lib/LLVMgold.so
CFLAGS=-O3
CC=clang -flto
-Wl,--plugin=/home/wer/llvm-3.0/build_config_gold_release/Release/lib/LLVMgold.so
RANLIB=/bin/true

Then configured and did the build for LLVM. While building Clang tool,
there may be large part of LLVM/Clang libraries which are not used by
Clang. So I was expecting appreciable reduction in size for Clang.
However memory savings were minimal.

Also, as explained in [2] in the topic "Compiler driver invokes link
time optimizer separately." , there are many cases when invoking Clang
with -flto may not remove the dead code. However I cannot figure out
how to build the Clang with first compiling all files to LLVM bit code
and then allow linker to treat them as compiled binaries which can be
linked together with LTO pass.

If this is possible by changes in Makefiles or build system, it can
help me a lot in removing some generic parts in Clang too to make a
less generic but lighter compiler tool.

Please help me with the same.

Thanks,
Ankur

[1] http://llvm.org/docs/GoldPlugin.html
[2] http://llvm.org/docs/LinkTimeOptimization.html

I guess I miss-interpreted the text in [2]. It talks about the
optimizations are hindered if the compiler driver invokes link time
optimizer "separately".

I found that all files compiled are in bc format. Also in library
archives, the embedded files were in bitcode format. So the gold
linker and LLVMgold plugin are indeed working fine.

However, there is still question over only 6% gain in the entire
process. Can someone help explaining reason for the same. I was
expecting large part of LLVM libraries may not be used in Clang tool
binary.

Thanks,
Ankur

On Thu, May 24, 2012 at 10:04 AM, ankur deshwal <a.s.deshwal at gmail.com>
wrote:
> Hi all,
>
> I was trying to use LTO facility of LLVM to reduce the footprint of
> Clang itself. I build the ld-gold  and LLVMgold.so as described at [1]
> and then set the environment as described too. However, had to add the
> path for plugin manually as Clang was not able pass it to ld
> automatically. Following is the setting I used before starting to
> build (small foot Clang.
>
> CXX=clang++ -flto
>
-Wl,--plugin=/home/wer/llvm-3.0/build_config_gold_release/Release/lib/LLVMgold.so
> CFLAGS=-O3
> CC=clang -flto
-Wl,--plugin=/home/wer/llvm-3.0/build_config_gold_release/Release/lib/LLVMgold.so
> RANLIB=/bin/true
>
> Then configured and did the build for LLVM. While building Clang tool,
> there may be large part of LLVM/Clang libraries which are not used by
> Clang. So I was expecting appreciable reduction in size for Clang.
> However memory savings were minimal.
>
> Also, as explained in [2] in the topic "Compiler driver invokes link
> time optimizer separately." , there are many cases when invoking Clang
> with -flto may not remove the dead code. However I cannot figure out
> how to build the Clang with first compiling all files to LLVM bit code
> and then allow linker to treat them as compiled binaries which can be
> linked together with LTO pass.
>
> If this is possible by changes in Makefiles or build system, it can
> help me a lot in removing some generic parts in Clang too to make a
> less generic but lighter compiler tool.
>
> Please help me with the same.
>
> Thanks,
> Ankur
>
> [1] http://llvm.org/docs/GoldPlugin.html
> [2] http://llvm.org/docs/LinkTimeOptimization.html

> However, there is still question over only 6% gain in the entire
> process. Can someone help explaining reason for the same. I was
> expecting large part of LLVM libraries may not be used in Clang tool
> binary.
I think a good part of it is we passing -export-dynamic to the linker.
This tells it that a plugin could potentially use any symbol in clang.
Try linking clang again, but remove the -rdynamic option from the link
line.
> Thanks,
> Ankur
Cheers,
Rafael