llvm dev - Mar 2018 - Commit module to Git after each Pass

Hello,

I had a stupid idea recently that turned out not so stupid after all. I
wanted to be able to "see" an entire pass pipeline in action to find
unnecessary transformations and/or missed opportunities and generally
improve the debug-ability of LLVM.

So as the title suggest, I implemented an equivalent of
"-print-after-all"
but instead of printing into stdout I dump into a file that get commit into
a temporary git. There are some quirks with it but it's working and is
actually awesome. For example, at first sight, I see multiple time lcssa
and instcombine cancelling each other's work.

Of course, that has a big impact on compile time when enabled, but that's
still practical (git being quite good at its job) when debugging.

There are improvement I can make, but would you guys be interested in such
feature?

-- 
*Alexandre Isoard*
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180306/a7bd1e89/attachment.html>

On 6 March 2018 at 18:43, Alexandre Isoard via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> I had a stupid idea recently that turned out not so stupid after all. I
> wanted to be able to "see" an entire pass pipeline in action to
find
> unnecessary transformations and/or missed opportunities and generally
> improve the debug-ability of LLVM.
>
> So as the title suggest, I implemented an equivalent of
"-print-after-all"
> but instead of printing into stdout I dump into a file that get commit into
> a temporary git. There are some quirks with it but it's working and is
> actually awesome. For example, at first sight, I see multiple time lcssa
and
> instcombine cancelling each other's work.
>
> Of course, that has a big impact on compile time when enabled, but
that's
> still practical (git being quite good at its job) when debugging.
>
> There are improvement I can make, but would you guys be interested in such
> feature?
Hi Alexandre. I can definitely see how it could be useful to track
changes through git commits, and take advantage of your favourite repo
history viewer to see changes. How much of your current implementation
is handled via modifications to LLVM vs an external helper script? For
instance, I might imagine trying to achieve something similar through
a script that parses the output of -print-after-all in order to create
the desired files+commits.

Best,

Alex

Today it is entirely in llvm. It is even more costly than -print-after-all
as it:
  - print to file
  - print the entire module (after each basic block, for basic block
passes, it will still print the entire module where only one basic block
changed)
  - call git 2 times (add then commit) and wait for them to finish (I even
save all the empty commits)

The reason I print the entire module is so that git is able to
show/compress the change properly. Then a simple git log --patch show the
change of each pass.

Ideally, the -debug output of each pass would be piped as the git commit
message, and the passes name+options would be the commit title. But I
didn't have time to do that.

My goal was not particularly to be efficient but to be thorough so as to
preserve the maximum of behavioral information of each pass while not
affecting their order and behavior.

You are right, there could be major speed improvement gained by doing so as
external processing. For instance, the output would be in the format that
git patch takes as input. Could this be a GSoC?

Today I use git filter-branch + opt as a post processing tool to generate
the reg.dot files of the region info. In any way, the feature is a breath
of fresh air in our development here. That's why I wanted to share. It will
quickly become my reflex to debug with it... probably because I am at ease
with git though.

On Wed, Mar 7, 2018, 06:19 Alex Bradbury <asb at asbradbury.org> wrote:
> On 6 March 2018 at 18:43, Alexandre Isoard via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > Hello,
> >
> > I had a stupid idea recently that turned out not so stupid after all.
I
> > wanted to be able to "see" an entire pass pipeline in action
to find
> > unnecessary transformations and/or missed opportunities and generally
> > improve the debug-ability of LLVM.
> >
> > So as the title suggest, I implemented an equivalent of
> "-print-after-all"
> > but instead of printing into stdout I dump into a file that get commit
> into
> > a temporary git. There are some quirks with it but it's working
and is
> > actually awesome. For example, at first sight, I see multiple time
lcssa
> and
> > instcombine cancelling each other's work.
> >
> > Of course, that has a big impact on compile time when enabled, but
that's
> > still practical (git being quite good at its job) when debugging.
> >
> > There are improvement I can make, but would you guys be interested in
> such
> > feature?
>
> Hi Alexandre. I can definitely see how it could be useful to track
> changes through git commits, and take advantage of your favourite repo
> history viewer to see changes. How much of your current implementation
> is handled via modifications to LLVM vs an external helper script? For
> instance, I might imagine trying to achieve something similar through
> a script that parses the output of -print-after-all in order to create
> the desired files+commits.
>
> Best,
>
> Alex
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180307/0405b928/attachment.html>

This is interesting, and might be useful.  I don't know that this is 
broadly useful enough for upstream inclusion, but if you could post this 
to github somewhere, I might play with it.

There might also be room to factor out common functionality. We've also 
run into the need to print whole-module instead of containing construct 
(i.e. this loop).  If we added upstream support for something along the 
lines of -print-module-after-all, building the git history could easily 
be done as a post processing step.

Philip


On 03/06/2018 10:43 AM, Alexandre Isoard via llvm-dev
wrote:
> Hello,
>
> I had a stupid idea recently that turned out not so stupid after all. 
> I wanted to be able to "see" an entire pass pipeline in action to
find
> unnecessary transformations and/or missed opportunities and generally 
> improve the debug-ability of LLVM.
>
> So as the title suggest, I implemented an equivalent of 
> "-print-after-all" but instead of printing into stdout I dump
into a
> file that get commit into a temporary git. There are some quirks with 
> it but it's working and is actually awesome. For example, at first 
> sight, I see multiple time lcssa and instcombine cancelling each 
> other's work.
>
> Of course, that has a big impact on compile time when enabled, but 
> that's still practical (git being quite good at its job) when
debugging.
>
> There are improvement I can make, but would you guys be interested in 
> such feature?
>
> -- 
> *Alexandre Isoard*
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180314/40d81d8d/attachment.html>

On 14 March 2018 at 20:51, Philip Reames via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> This is interesting, and might be useful.  I don't know that this is
broadly
> useful enough for upstream inclusion, but if you could post this to github
> somewhere, I might play with it.
>
> There might also be room to factor out common functionality.  We've
also run
> into the need to print whole-module instead of containing construct (i.e.
> this loop).  If we added upstream support for something along the lines of
> -print-module-after-all, building the git history could easily be done as a
> post processing step.
Alexandre posted the patch here https://reviews.llvm.org/D44244

I made a similar suggestion regarding handling this as a
post-processing step. I like your proposed -print-module-after-all
name.

Best,

Alex

The print-module-after-all type of option exists in upstream:
  -print-module-scope                                            - When printing
IR for print-[before|after]{-all} always print a module IR

commit 7d160f714357f6784ead669ce516e94991c12e5a
Author: Fedor Sergeev <fedor.sergeev at azul.com<mailto:fedor.sergeev at
azul.com>>
Date:   Fri Dec 1 17:42:46 2017 +0000

    IR printing improvement for function passes - introducing
-print-module-scope



    Summary:
    When debugging function passes it happens to be rather useful to dump
    the whole module before the transformation and then use this dump
    to analyze this single transformation by running it separately
    on that particular module state.



    Introducing
        -print-module-scope
    debugging option that forces all the function-level IR dumps
    to become whole-module dumps.



    This option builds on top of normal dumping controls like
       -print-before/after
       -filter-print-funcs



    The plan is to eventually extend this option to cover other local passes
    (at least loop passes) but that should go as a separate change.


Loop passes here:
commit 5608259c999fb77c5d6093895696f4daebe6b8cd
Author: Fedor Sergeev <fedor.sergeev at azul.com<mailto:fedor.sergeev at
azul.com>>
Date:   Fri Dec 1 18:33:58 2017 +0000

    IR printing improvement for loop passes - handle -print-module-scope


-Daniel


On Mar 14, 2018, at 3:51 PM, Philip Reames via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:


This is interesting, and might be useful.  I don't know that this is broadly
useful enough for upstream inclusion, but if you could post this to github
somewhere, I might play with it.

There might also be room to factor out common functionality.  We've also run
into the need to print whole-module instead of containing construct (i.e. this
loop).  If we added upstream support for something along the lines of
-print-module-after-all, building the git history could easily be done as a post
processing step.

Philip

On 03/06/2018 10:43 AM, Alexandre Isoard via llvm-dev wrote:
Hello,

I had a stupid idea recently that turned out not so stupid after all. I wanted
to be able to "see" an entire pass pipeline in action to find
unnecessary transformations and/or missed opportunities and generally improve
the debug-ability of LLVM.

So as the title suggest, I implemented an equivalent of
"-print-after-all" but instead of printing into stdout I dump into a
file that get commit into a temporary git. There are some quirks with it but
it's working and is actually awesome. For example, at first sight, I see
multiple time lcssa and instcombine cancelling each other's work.

Of course, that has a big impact on compile time when enabled, but that's
still practical (git being quite good at its job) when debugging.

There are improvement I can make, but would you guys be interested in such
feature?

--
Alexandre Isoard



_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180314/3a663688/attachment-0001.html>

+1   Being able to look at the diffs of the dumps after passes sounds 
really nice. /Jonas


On 2018-03-14 21:51, Philip Reames via llvm-dev wrote:
>
> This is interesting, and might be useful.  I don't know that this is 
> broadly useful enough for upstream inclusion, but if you could post 
> this to github somewhere, I might play with it.
>
> There might also be room to factor out common functionality. We've 
> also run into the need to print whole-module instead of containing 
> construct (i.e. this loop).  If we added upstream support for 
> something along the lines of -print-module-after-all, building the git 
> history could easily be done as a post processing step.
>
> Philip
>
>
> On 03/06/2018 10:43 AM, Alexandre Isoard via llvm-dev wrote:
>> Hello,
>>
>> I had a stupid idea recently that turned out not so stupid after all. 
>> I wanted to be able to "see" an entire pass pipeline in
action to
>> find unnecessary transformations and/or missed opportunities and 
>> generally improve the debug-ability of LLVM.
>>
>> So as the title suggest, I implemented an equivalent of 
>> "-print-after-all" but instead of printing into stdout I dump
into a
>> file that get commit into a temporary git. There are some quirks with 
>> it but it's working and is actually awesome. For example, at first 
>> sight, I see multiple time lcssa and instcombine cancelling each 
>> other's work.
>>
>> Of course, that has a big impact on compile time when enabled, but 
>> that's still practical (git being quite good at its job) when
debugging.
>>
>> There are improvement I can make, but would you guys be interested in 
>> such feature?
>>
>> -- 
>> *Alexandre Isoard*
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180316/11e864d1/attachment.html>