llvm dev - Jun 2021 - RFC: Reducing metadata in LLVM tests

Hi Djordje,

I think something like this would be super useful. Can you explain how it
differs from the metadata reduction in bugpoint and to what degree the two share
(or could share) code?

-- adrian
> On Jun 4, 2021, at 7:06 AM, Djordje Todorovic <djordje.todorovic at
syrmia.com> wrote:
> 
> Hi,
> 
> We are seeing that the LLVM test suite grows every day, so there is always
a need for reducing the new tests as well as the old ones. There are tools
designed to do that, e.g. [0] and [1]. When an IR (or MIR) test contains Debug
Info, the LLVM DI Metadata makes the test obviously longer. Not all of these
metadata are always necessary for the test, so reviewers frequently ask patch
submitters for a reduced test case in terms of DI Metadata by recommending some
tricks described in [2]. The [2] is just a small proposal for a utility tool
that will save us some time (when fully implemented) during making the patches
as well as when doing code reviews -- any thoughts on this?
> 
> [0] https://llvm.org/docs/CommandGuide/bugpoint.html
<https://llvm.org/docs/CommandGuide/bugpoint.html>
> [1] https://blog.regehr.org/archives/2109
<https://blog.regehr.org/archives/2109>
> [2] https://github.com/djolertrk/llvm-metadataburn
<https://github.com/djolertrk/llvm-metadataburn>
> 
> Best regards,
> Djordje
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210604/bd2eb24e/attachment.html>

Hi Adrian,

I'm not opposed to adding something like this into the llvm-reduce
(ReduceMetadata) utility, and I think it will be a better way.
@Jeremy Morse<mailto:jeremy.morse at sony.com> has mentioned that someone
from SONY is working on something like this -- I am happy with that and I just
want to avoid redundant work.

Best regards,
Djordje

________________________________
From: Adrian Prantl <aprantl at apple.com>
Sent: Friday, June 4, 2021 7:42 PM
To: Djordje Todorovic <Djordje.Todorovic at syrmia.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>; Jeremy Morse
<jeremy.morse at sony.com>; Petar Jovanovic <petar.jovanovic at
syrmia.com>; asowda at cisco.com <asowda at cisco.com>; ibaev at
cisco.com <ibaev at cisco.com>
Subject: Re: RFC: Reducing metadata in LLVM tests

Hi Djordje,

I think something like this would be super useful. Can you explain how it
differs from the metadata reduction in bugpoint and to what degree the two share
(or could share) code?

-- adrian

On Jun 4, 2021, at 7:06 AM, Djordje Todorovic <djordje.todorovic at
syrmia.com<mailto:djordje.todorovic at syrmia.com>> wrote:

Hi,

We are seeing that the LLVM test suite grows every day, so there is always a
need for reducing the new tests as well as the old ones. There are tools
designed to do that, e.g. [0] and [1]. When an IR (or MIR) test contains Debug
Info, the LLVM DI Metadata makes the test obviously longer. Not all of these
metadata are always necessary for the test, so reviewers frequently ask patch
submitters for a reduced test case in terms of DI Metadata by recommending some
tricks described in [2]. The [2] is just a small proposal for a utility tool
that will save us some time (when fully implemented) during making the patches
as well as when doing code reviews -- any thoughts on this?

[0] https://llvm.org/docs/CommandGuide/bugpoint.html
[1] https://blog.regehr.org/archives/2109
[2] https://github.com/djolertrk/llvm-metadataburn

Best regards,
Djordje

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210607/94f8c3b8/attachment.html>

My main concern would be that significantly automatically reduced LLVM
debug info IR metadata might produce "strange" examples (valid
according to the verifier, but so quirky as to be hard to know if
they're meant to be supported or if we should/might eventually deem
them invalid according to the verifier) that may be hard to
understand, while still not being short enough to be hand craftable -
in which case I'm not sure how much of an improvement to the current
situation they would be.

So I think at least for the first few examples of automatically (or
even significantly hand reduced) test cases with debug info metadata
we'll want to look pretty carefully at them to discuss whether they
improve or harm test case maintainability.

On Fri, Jun 4, 2021 at 10:42 AM Adrian Prantl via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hi Djordje,
>
> I think something like this would be super useful. Can you explain how it
differs from the metadata reduction in bugpoint and to what degree the two share
(or could share) code?
>
> -- adrian
>
> On Jun 4, 2021, at 7:06 AM, Djordje Todorovic <djordje.todorovic at
syrmia.com> wrote:
>
> Hi,
>
> We are seeing that the LLVM test suite grows every day, so there is always
a need for reducing the new tests as well as the old ones. There are tools
designed to do that, e.g. [0] and [1]. When an IR (or MIR) test contains Debug
Info, the LLVM DI Metadata makes the test obviously longer. Not all of these
metadata are always necessary for the test, so reviewers frequently ask patch
submitters for a reduced test case in terms of DI Metadata by recommending some
tricks described in [2]. The [2] is just a small proposal for a utility tool
that will save us some time (when fully implemented) during making the patches
as well as when doing code reviews -- any thoughts on this?
>
> [0] https://llvm.org/docs/CommandGuide/bugpoint.html
> [1] https://blog.regehr.org/archives/2109
> [2] https://github.com/djolertrk/llvm-metadataburn
>
> Best regards,
> Djordje
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev