llvm dev - Oct 2019 - [RFC] Adding time-trace to LLD?

Hi all,

I've been using -ftime-trace on the compiler and find it very useful for
analysing compile times and whole build times. I've made a first attempt at
adding this to LLD (ELF) so link times can be traced in the same way:
https://reviews.llvm.org/D69043

This works for LTO and ThinLTO, but needs some more work to make sure it
scales well across multiple threads.

Please let me know whether you think this is worth pursuing?

Thanks
Russ
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191016/4b4e4215/attachment.html>

The COFF flavor of lld has an option to show basic timing information on
stdout: https://llvm.org/r322736. I enabled it on my project's CI and find
it quite useful in logs. I'd be happy to see it ported to ELF.
>From looking at the screenshot on your review, it looks like your proposal
provides much richer information, but the extra friction of opening a json
file (and pulling it off our builders) means I'd be less inclined to reach
for this feature unless I was debugging a specific issue. I guess it
depends on whether you're going for day-to-day health monitoring or
targeted problem-solving.



On Wed, Oct 16, 2019 at 11:34 AM Russell Gallop via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi all,
>
> I've been using -ftime-trace on the compiler and find it very useful
for
> analysing compile times and whole build times. I've made a first
attempt at
> adding this to LLD (ELF) so link times can be traced in the same way:
> https://reviews.llvm.org/D69043
>
> This works for LTO and ThinLTO, but needs some more work to make sure it
> scales well across multiple threads.
>
> Please let me know whether you think this is worth pursuing?
>
> Thanks
> Russ
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191016/f20ea417/attachment.html>

Hi David,

Thanks for the feedback.
> The COFF flavor of lld has an option to show basic timing information on
stdout: https://llvm.org/r322736. I enabled it on my project's CI and find
it quite useful in logs. I'd be happy to see it ported to ELF.

It's interesting to compare. They're quite similar in implementation:
using
scoped timers. I wonder whether it would be possible to combine them with
options to control level of tracing (fine or coarse), and how the output is
produced (stdout or JSON). I think that would require more than just a time
granularity control, maybe an indication of "level" on each trace
analogous to logging levels.

One thing I like about the approach I propose here is that it is shared
between the compiler and linker. It shares the timer implementation and
makes the output more consistent for someone analysing a whole build.
> From looking at the screenshot on your review, it looks like your
proposal provides much richer information, but the extra friction of
opening a json file (and pulling it off our builders) means I'd be less
inclined to reach for this feature unless I was debugging a specific
issue.
> I guess it depends on whether you're going for day-to-day health
monitoring or targeted problem-solving.

I think you're suggesting /TIME is better for day-to-day health monitoring
and -time-trace for targeted problem-solving. I agree they have different
strengths. On the compiler side I find that the JSON format is helpful for
monitoring as well, as it's easier to automate checks and analysis from a
machine readable format. Whatever approach does need to be low enough
overhead to be useful in that way.

Regards
Russ

On Wed, 16 Oct 2019 at 18:15, David Major <dmajor at mozilla.com> wrote:
> The COFF flavor of lld has an option to show basic timing information on
> stdout: https://llvm.org/r322736. I enabled it on my project's CI and
> find it quite useful in logs. I'd be happy to see it ported to ELF.
>
> From looking at the screenshot on your review, it looks like your proposal
> provides much richer information, but the extra friction of opening a json
> file (and pulling it off our builders) means I'd be less inclined to
reach
> for this feature unless I was debugging a specific issue. I guess it
> depends on whether you're going for day-to-day health monitoring or
> targeted problem-solving.
>
>
>
> On Wed, Oct 16, 2019 at 11:34 AM Russell Gallop via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi all,
>>
>> I've been using -ftime-trace on the compiler and find it very
useful for
>> analysing compile times and whole build times. I've made a first
attempt at
>> adding this to LLD (ELF) so link times can be traced in the same way:
>> https://reviews.llvm.org/D69043
>>
>> This works for LTO and ThinLTO, but needs some more work to make sure
it
>> scales well across multiple threads.
>>
>> Please let me know whether you think this is worth pursuing?
>>
>> Thanks
>> Russ
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191017/dbfab2b5/attachment.html>