llvm dev - Apr 2021 - [RFC] Generating LLD reproducers on crashes

On 2021-04-14, Haowei Wu via llvm-dev wrote:
>*Background / Motivation*
>
>Both clang and lld have the ability to generate a reproducer (an archive
>with input files and invoker script to reproduce the clang/lld build).
>While clang will generate a reproducer archive when a crash happens, lld
>only generates a reproducer when '--reproduce' flag is explicitly
set (this
>is equivalent to Clang's -gen-reproducer flag). This is not very helpful
>for debugging lld bugs, particularly when the crash happens in building big
>projects, since it will be unrealistic to set reproducer flags to generate
>reproducer archives for every lld invocation. This design also causes
>troubles when the crash happens on bots only, as in most cases, developers
>do not have access to the file system of these bots. It would be great to
>improve the lld reproducer generation for easier debugging in these
>scenarios.
>
>*Proposal*
>
>Given the use cases and status of clang and lld. I think there are 2
>possible solutions.
>
>*Extend Clang driver*
>In most cases, lld is invoked by the clang driver instead of being invoked
>by the build system directly. Therefore, the clang driver can be changed to
>re-invoke lld with '--reproduce' flags when it detects the lld
subprocess
>is crashed.
>
>Advantages:
>    * It probably does not require any changes to the lld and might be
>easier than handling the crash directly in lld.
>
>Disadvantages:
>    * In case when there is a racing condition in the build system, the
>input files might have changed between 1st lld crash and 2nd lld rerun with
>'--reproduce' flag. In this case, the generated lld reproducer
archive
>might not be able to trigger a crash, makes it less useful.
>
>*Improve lld reproducer*
>Another way would be to make lld generate a reproducer archive when it
>crashes, just like what clang is doing.
>
>Advantages:
>    * It will work no matter if lld is invoked from Clang or from the build
>system.
>    * It will catch the input file in case the crash is caused by build
>races.
>
>Disadvantages:
>    * It might need a lot of work if lld does not already have a
>sophisticated crash handler. It might still need some plumbing changes in
>clang driver so lld can honor the '-fcrash-diagnostic-dir' flag.
>
>*Comments?*
>Which approach do you prefer? Feel free to share your opinions.
There is a resource difference between clang -gen-reproducer /
environment variable "FORCE_CLANG_DIAGNOSTICS_CRASH" and ld.lld
--reproduce.

clang -gen-reproducer produces a source file and a .sh file for one
single translation unit, the space consumption is low.
ld.lld --reproduce can potentially pack a large list of files, which may
take hundreds of megabytes or several gigabytes.

I am skeptical that users will want to have this behavior by default.
If this behavior is guarded by an option, it might be fine.

> I am skeptical that users will want to have this behavior by default.
> If this behavior is guarded by an option, it might be fine.
That's a good point. If the reproducer will be more than a few hundreds
MiBs, it is definitely not suitable to be enabled by default. I agree it's
better to be guarded by an option flag such as `--gen-lld-crash-reproducer`.

On Wed, Apr 14, 2021 at 2:40 PM Fangrui Song <maskray at google.com>
wrote:
>
> On 2021-04-14, Haowei Wu via llvm-dev wrote:
> >*Background / Motivation*
> >
> >Both clang and lld have the ability to generate a reproducer (an
archive
> >with input files and invoker script to reproduce the clang/lld build).
> >While clang will generate a reproducer archive when a crash happens,
lld
> >only generates a reproducer when '--reproduce' flag is
explicitly set
> (this
> >is equivalent to Clang's -gen-reproducer flag). This is not very
helpful
> >for debugging lld bugs, particularly when the crash happens in building
> big
> >projects, since it will be unrealistic to set reproducer flags to
generate
> >reproducer archives for every lld invocation. This design also causes
> >troubles when the crash happens on bots only, as in most cases,
developers
> >do not have access to the file system of these bots. It would be great
to
> >improve the lld reproducer generation for easier debugging in these
> >scenarios.
> >
> >*Proposal*
> >
> >Given the use cases and status of clang and lld. I think there are 2
> >possible solutions.
> >
> >*Extend Clang driver*
> >In most cases, lld is invoked by the clang driver instead of being
invoked
> >by the build system directly. Therefore, the clang driver can be
changed
> to
> >re-invoke lld with '--reproduce' flags when it detects the lld
subprocess
> >is crashed.
> >
> >Advantages:
> >    * It probably does not require any changes to the lld and might be
> >easier than handling the crash directly in lld.
> >
> >Disadvantages:
> >    * In case when there is a racing condition in the build system, the
> >input files might have changed between 1st lld crash and 2nd lld rerun
> with
> >'--reproduce' flag. In this case, the generated lld reproducer
archive
> >might not be able to trigger a crash, makes it less useful.
> >
> >*Improve lld reproducer*
> >Another way would be to make lld generate a reproducer archive when it
> >crashes, just like what clang is doing.
> >
> >Advantages:
> >    * It will work no matter if lld is invoked from Clang or from the
> build
> >system.
> >    * It will catch the input file in case the crash is caused by build
> >races.
> >
> >Disadvantages:
> >    * It might need a lot of work if lld does not already have a
> >sophisticated crash handler. It might still need some plumbing changes
in
> >clang driver so lld can honor the '-fcrash-diagnostic-dir'
flag.
> >
> >*Comments?*
> >Which approach do you prefer? Feel free to share your opinions.
>
> There is a resource difference between clang -gen-reproducer /
> environment variable "FORCE_CLANG_DIAGNOSTICS_CRASH" and ld.lld
> --reproduce.
>
> clang -gen-reproducer produces a source file and a .sh file for one
> single translation unit, the space consumption is low.
> ld.lld --reproduce can potentially pack a large list of files, which may
> take hundreds of megabytes or several gigabytes.
>
> I am skeptical that users will want to have this behavior by default.
> If this behavior is guarded by an option, it might be fine.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210414/d4511cd4/attachment.html>