llvm dev - Jun 2019 - LLD/Link find out what triggers 80mb exe

Unfortunately, lld link doesn't do -map yet (and link.exe /map is kinda
useless; it shows what get linked in, but only refers to an object file, not
function name when it does so, like a single reference in that object file
triggers all it's dependencies)

Yeah I'm fairly sure it's caused by something I'm doing wrong, I
just have no idea how to find out. I thought -function-section would have pretty
much covered it.

On Fri, Jun 7, 2019, at 14:22, Rui Ueyama wrote:
> If you use only one function from a static library, and that adds 80 MiB to
the executable, it is clear that that undefined symbol causes a lot of other
files to be loaded, so the question is why that function has such a large
(transitive) dependency and how to find the cause.
> 
> Unfortunately there's no exact command line option or a feature to find
it out. I have an idea to write a post-link analysis tool, which processes a
linker's debug output, to find out a "weak link" in a dependency
graph. If we can split a graph into two by removing a small number of symbols
(or vertices) from the graph, we can reduce a program size by removing these
symbols from the program. But that kind of analysis tool is not available yet,
and I don't know how powerful it would actually be.
> 
> There are a few options that may be useful when debugging the issue.
> 
> `-v` makes lld to print out a file name as the linker loads a new file.
> 
> `-Map=<filename>` makes lld to write an output file layout
information to a given file.
> 
> On Fri, Jun 7, 2019 at 8:59 PM Carlo Kok via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>> Is there any way to find out what causes a 80mb executable with
lld/link?
>> 
>> 
>>  The object files are compiled with -function-sections and
-data-sections, the originating .lib is 270mb, but I'm calling 1 function in
it, which might call some stuff recursively, but I don't know how to find
out what exactly triggers the use. Is there a commandline option that shows
this?
>> 
>>  (repro is 286 mb) : http://gofile.me/4Gflq/OOr8WKU12
>>  _______________________________________________
>>  LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190607/3b76f7e8/attachment.html>

Just to double check: are you calling lld-link with `-opt:ref`? It may or
may not be enabled by default, depending on other flags.


On Fri, Jun 7, 2019 at 8:26 AM Carlo Kok via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Unfortunately, lld link doesn't do -map yet (and link.exe /map is kinda
> useless; it shows what get linked in, but only refers to an object file,
> not function name when it does so, like a single reference in that object
> file triggers all it's dependencies)
>
> Yeah I'm fairly sure it's caused by something I'm doing wrong,
I just have
> no idea how to find out. I thought -function-section would have pretty much
> covered it.
>
> On Fri, Jun 7, 2019, at 14:22, Rui Ueyama wrote:
>
> If you use only one function from a static library, and that adds 80 MiB
> to the executable, it is clear that that undefined symbol causes a lot of
> other files to be loaded, so the question is why that function has such a
> large (transitive) dependency and how to find the cause.
>
> Unfortunately there's no exact command line option or a feature to find
it
> out. I have an idea to write a post-link analysis tool, which processes a
> linker's debug output, to find out a "weak link" in a
dependency graph. If
> we can split a graph into two by removing a small number of symbols (or
> vertices) from the graph, we can reduce a program size by removing these
> symbols from the program. But that kind of analysis tool is not available
> yet, and I don't know how powerful it would actually be.
>
> There are a few options that may be useful when debugging the issue.
>
> `-v` makes lld to print out a file name as the linker loads a new file.
>
> `-Map=<filename>` makes lld to write an output file layout
information to
> a given file.
>
> On Fri, Jun 7, 2019 at 8:59 PM Carlo Kok via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Is there any way to find out what causes a 80mb executable with lld/link?
>
>
> The object files are compiled with -function-sections and -data-sections,
> the originating .lib is 270mb, but I'm calling 1 function in it, which
> might call some stuff recursively, but I don't know how to find out
what
> exactly triggers the use. Is there a commandline option that shows this?
>
> (repro is 286 mb) : http://gofile.me/4Gflq/OOr8WKU12
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190607/4dca6de7/attachment.html>

I did try that yes. Didn't seem to make any difference (still 80ishmb)

On Fri, Jun 7, 2019, at 15:29, David Major wrote:
> Just to double check: are you calling lld-link with `-opt:ref`? It may or
may not be enabled by default, depending on other flags.
> 
> 
> On Fri, Jun 7, 2019 at 8:26 AM Carlo Kok via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>> __
>> Unfortunately, lld link doesn't do -map yet (and link.exe /map is
kinda useless; it shows what get linked in, but only refers to an object file,
not function name when it does so, like a single reference in that object file
triggers all it's dependencies)
>> 
>> Yeah I'm fairly sure it's caused by something I'm doing
wrong, I just have no idea how to find out. I thought -function-section would
have pretty much covered it.
>> 
>> On Fri, Jun 7, 2019, at 14:22, Rui Ueyama wrote:
>>> If you use only one function from a static library, and that adds
80 MiB to the executable, it is clear that that undefined symbol causes a lot of
other files to be loaded, so the question is why that function has such a large
(transitive) dependency and how to find the cause.
>>> 
>>> Unfortunately there's no exact command line option or a feature
to find it out. I have an idea to write a post-link analysis tool, which
processes a linker's debug output, to find out a "weak link" in a
dependency graph. If we can split a graph into two by removing a small number of
symbols (or vertices) from the graph, we can reduce a program size by removing
these symbols from the program. But that kind of analysis tool is not available
yet, and I don't know how powerful it would actually be.
>>> 
>>> There are a few options that may be useful when debugging the
issue.
>>> 
>>> `-v` makes lld to print out a file name as the linker loads a new
file.
>>> 
>>> `-Map=<filename>` makes lld to write an output file layout
information to a given file.
>>> 
>>> On Fri, Jun 7, 2019 at 8:59 PM Carlo Kok via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>>> Is there any way to find out what causes a 80mb executable with
lld/link?
>>>> 
>>>> 
>>>> The object files are compiled with -function-sections and
-data-sections, the originating .lib is 270mb, but I'm calling 1 function in
it, which might call some stuff recursively, but I don't know how to find
out what exactly triggers the use. Is there a commandline option that shows
this?
>>>> 
>>>> (repro is 286 mb) : http://gofile.me/4Gflq/OOr8WKU12
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> _______________________________________________
>>  LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190607/bfc965bd/attachment.html>