llvm dev - Jan 2020 - Where does LTO remove unused functions?

Hi!

LLVM newbie here, I was mainly working on the frontend so far. We had a
small hackathon project idea to piggyback on LTO to detect dead code
(unused functions). The basic idea is to compile the code for every target
and dump the removed functions. Intersect the function symbol names for
each target and those functions should be safe to remove from the source
code (unless there were some configuration that was not compiled). Is this
reasonable?

I started to play around with some toy examples and got stuck on the very
beginning not being able to figure out where the unused functions are
actually getting removed.

Here is what I did:
tu1.cpp:
int unused(int a);
int probably_inlined(int a);
int main(int argc, const char *argv[]) {
  return probably_inlined(argc);
}

tu2.cpp:
int unused(int a) {
  return a + 1;
}
int probably_inlined(int a) {
  return a + 2;
}

I produced two object files with bitcode:
clang -c -flto tu1.cpp -o tu1.o

And I run LTO and attempted to dump the IR before each pass:
clang -O2 -Wl,-mllvm -Wl,-print-before-all tu1.o tu2.o -o optimized

In my dumps I saw the function unused removed even before the first pass.
Where did that happen? I tried to invoke llvm-link manually and that did
not remove unused.

I also tried to dump optimization remarks and was no trace of a function
being removed (I only saw a function being inlined).

Thanks in advance,
Gabor
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200127/01c1a1c6/attachment-0001.html>

By default even regular LTO now has module summaries (like the kind used
for ThinLTO). LTO will then run index based dead symbol analysis here:
http://llvm-cs.pcc.me.uk/lib/LTO/LTO.cpp#923. Then when linkRegularLTO is
called here: http://llvm-cs.pcc.me.uk/lib/LTO/LTO.cpp#935, it indicates
that the index should be consulted for liveness, and that routine skips
even adding the dead symbols to the Keep set. So they never make it into
the combined module.

Teresa

On Mon, Jan 27, 2020 at 5:43 PM Gábor Horváth via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi!
>
> LLVM newbie here, I was mainly working on the frontend so far. We had a
> small hackathon project idea to piggyback on LTO to detect dead code
> (unused functions). The basic idea is to compile the code for every target
> and dump the removed functions. Intersect the function symbol names for
> each target and those functions should be safe to remove from the source
> code (unless there were some configuration that was not compiled). Is this
> reasonable?
>
> I started to play around with some toy examples and got stuck on the very
> beginning not being able to figure out where the unused functions are
> actually getting removed.
>
> Here is what I did:
> tu1.cpp:
> int unused(int a);
> int probably_inlined(int a);
> int main(int argc, const char *argv[]) {
>   return probably_inlined(argc);
> }
>
> tu2.cpp:
> int unused(int a) {
>   return a + 1;
> }
> int probably_inlined(int a) {
>   return a + 2;
> }
>
> I produced two object files with bitcode:
> clang -c -flto tu1.cpp -o tu1.o
>
> And I run LTO and attempted to dump the IR before each pass:
> clang -O2 -Wl,-mllvm -Wl,-print-before-all tu1.o tu2.o -o optimized
>
> In my dumps I saw the function unused removed even before the first pass.
> Where did that happen? I tried to invoke llvm-link manually and that did
> not remove unused.
>
> I also tried to dump optimization remarks and was no trace of a function
> being removed (I only saw a function being inlined).
>
> Thanks in advance,
> Gabor
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200127/d86327ce/attachment.html>

Thanks!

On Mon, Jan 27, 2020 at 6:05 PM Teresa Johnson <tejohnson at google.com>
wrote:
> By default even regular LTO now has module summaries (like the kind used
> for ThinLTO). LTO will then run index based dead symbol analysis here:
> http://llvm-cs.pcc.me.uk/lib/LTO/LTO.cpp#923. Then when linkRegularLTO is
> called here: http://llvm-cs.pcc.me.uk/lib/LTO/LTO.cpp#935, it indicates
> that the index should be consulted for liveness, and that routine skips
> even adding the dead symbols to the Keep set. So they never make it into
> the combined module.
>
> Teresa
>
> On Mon, Jan 27, 2020 at 5:43 PM Gábor Horváth via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi!
>>
>> LLVM newbie here, I was mainly working on the frontend so far. We had a
>> small hackathon project idea to piggyback on LTO to detect dead code
>> (unused functions). The basic idea is to compile the code for every
target
>> and dump the removed functions. Intersect the function symbol names for
>> each target and those functions should be safe to remove from the
source
>> code (unless there were some configuration that was not compiled). Is
this
>> reasonable?
>>
>> I started to play around with some toy examples and got stuck on the
very
>> beginning not being able to figure out where the unused functions are
>> actually getting removed.
>>
>> Here is what I did:
>> tu1.cpp:
>> int unused(int a);
>> int probably_inlined(int a);
>> int main(int argc, const char *argv[]) {
>>   return probably_inlined(argc);
>> }
>>
>> tu2.cpp:
>> int unused(int a) {
>>   return a + 1;
>> }
>> int probably_inlined(int a) {
>>   return a + 2;
>> }
>>
>> I produced two object files with bitcode:
>> clang -c -flto tu1.cpp -o tu1.o
>>
>> And I run LTO and attempted to dump the IR before each pass:
>> clang -O2 -Wl,-mllvm -Wl,-print-before-all tu1.o tu2.o -o optimized
>>
>> In my dumps I saw the function unused removed even before the first
>> pass. Where did that happen? I tried to invoke llvm-link manually and
>> that did not remove unused.
>>
>> I also tried to dump optimization remarks and was no trace of a
function
>> being removed (I only saw a function being inlined).
>>
>> Thanks in advance,
>> Gabor
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200127/bb64f428/attachment.html>

you can find unused functions easily using static analysis tools; codeql comes
to mind:

https://help.semmle.com/QL/learn-ql/cpp/function-classes.html


import cpp from Function f where not exists(FunctionCall fc | fc.getTarget() =
f) select f, "This function is never called."


Sent from my iPhone
> On Jan 27, 2020, at 19:43, Gábor Horváth via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi!
> 
> LLVM newbie here, I was mainly working on the frontend so far. We had a
small hackathon project idea to piggyback on LTO to detect dead code (unused
functions). The basic idea is to compile the code for every target and dump the
removed functions. Intersect the function symbol names for each target and those
functions should be safe to remove from the source code (unless there were some
configuration that was not compiled). Is this reasonable?
> 
> I started to play around with some toy examples and got stuck on the very
beginning not being able to figure out where the unused functions are actually
getting removed.
> 
> Here is what I did:
> tu1.cpp:
> int unused(int a);
> int probably_inlined(int a);
> int main(int argc, const char *argv[]) {
>   return probably_inlined(argc);
> }
> 
> tu2.cpp:
> int unused(int a) {
>   return a + 1;
> }
> int probably_inlined(int a) {
>   return a + 2;
> }
> 
> I produced two object files with bitcode:
> clang -c -flto tu1.cpp -o tu1.o
> 
> And I run LTO and attempted to dump the IR before each pass:
> clang -O2 -Wl,-mllvm -Wl,-print-before-all tu1.o tu2.o -o optimized
> 
> In my dumps I saw the function unused removed even before the first pass.
Where did that happen? I tried to invoke llvm-link manually and that did not
remove unused.
> 
> I also tried to dump optimization remarks and was no trace of a function
being removed (I only saw a function being inlined).
> 
> Thanks in advance,
> Gabor
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200128/1e12bb42/attachment.html>