Daniel Neilson via llvm-dev
2018-Mar-14 21:31 UTC
[llvm-dev] Commit module to Git after each Pass
The print-module-after-all type of option exists in upstream: -print-module-scope - When printing IR for print-[before|after]{-all} always print a module IR commit 7d160f714357f6784ead669ce516e94991c12e5a Author: Fedor Sergeev <fedor.sergeev at azul.com<mailto:fedor.sergeev at azul.com>> Date: Fri Dec 1 17:42:46 2017 +0000 IR printing improvement for function passes - introducing -print-module-scope Summary: When debugging function passes it happens to be rather useful to dump the whole module before the transformation and then use this dump to analyze this single transformation by running it separately on that particular module state. Introducing -print-module-scope debugging option that forces all the function-level IR dumps to become whole-module dumps. This option builds on top of normal dumping controls like -print-before/after -filter-print-funcs The plan is to eventually extend this option to cover other local passes (at least loop passes) but that should go as a separate change. Loop passes here: commit 5608259c999fb77c5d6093895696f4daebe6b8cd Author: Fedor Sergeev <fedor.sergeev at azul.com<mailto:fedor.sergeev at azul.com>> Date: Fri Dec 1 18:33:58 2017 +0000 IR printing improvement for loop passes - handle -print-module-scope -Daniel On Mar 14, 2018, at 3:51 PM, Philip Reames via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: This is interesting, and might be useful. I don't know that this is broadly useful enough for upstream inclusion, but if you could post this to github somewhere, I might play with it. There might also be room to factor out common functionality. We've also run into the need to print whole-module instead of containing construct (i.e. this loop). If we added upstream support for something along the lines of -print-module-after-all, building the git history could easily be done as a post processing step. Philip On 03/06/2018 10:43 AM, Alexandre Isoard via llvm-dev wrote: Hello, I had a stupid idea recently that turned out not so stupid after all. I wanted to be able to "see" an entire pass pipeline in action to find unnecessary transformations and/or missed opportunities and generally improve the debug-ability of LLVM. So as the title suggest, I implemented an equivalent of "-print-after-all" but instead of printing into stdout I dump into a file that get commit into a temporary git. There are some quirks with it but it's working and is actually awesome. For example, at first sight, I see multiple time lcssa and instcombine cancelling each other's work. Of course, that has a big impact on compile time when enabled, but that's still practical (git being quite good at its job) when debugging. There are improvement I can make, but would you guys be interested in such feature? -- Alexandre Isoard _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180314/3a663688/attachment-0001.html>
Fedor Sergeev via llvm-dev
2018-Mar-15 10:32 UTC
[llvm-dev] Commit module to Git after each Pass
For this to be really usable in this setup we need additionally to: - extend -print-module-scope to cover basic block passes - introduce a clear way to separate module IRs as those are being printed by -print-after-all But yes, it should work, and a wrapper that pipes to git fast-import seems to be the best way to handle it. regards, Fedor. On 03/15/2018 12:31 AM, Daniel Neilson via llvm-dev wrote:> The print-module-after-all type of option exists in upstream: > -print-module-scope - When > printing IR for print-[before|after]{-all} always print a module IR > > commit 7d160f714357f6784ead669ce516e94991c12e5a > Author: Fedor Sergeev <fedor.sergeev at azul.com > <mailto:fedor.sergeev at azul.com>> > Date: Fri Dec 1 17:42:46 2017 +0000 > > IR printing improvement for function passes - introducing > -print-module-scope > > > Summary: > When debugging function passes it happens to be rather useful to dump > the whole module before the transformation and then use this dump > to analyze this single transformation by running it separately > on that particular module state. > > > Introducing > -print-module-scope > debugging option that forces all the function-level IR dumps > to become whole-module dumps. > > > This option builds on top of normal dumping controls like > -print-before/after > -filter-print-funcs > > > The plan is to eventually extend this option to cover other local > passes > (at least loop passes) but that should go as a separate change. > > > Loop passes here: > commit 5608259c999fb77c5d6093895696f4daebe6b8cd > Author: Fedor Sergeev <fedor.sergeev at azul.com > <mailto:fedor.sergeev at azul.com>> > Date: Fri Dec 1 18:33:58 2017 +0000 > > IR printing improvement for loop passes - handle -print-module-scope > > > -Daniel > > >> On Mar 14, 2018, at 3:51 PM, Philip Reames via llvm-dev >> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >> >> This is interesting, and might be useful. I don't know that this is >> broadly useful enough for upstream inclusion, but if you could post >> this to github somewhere, I might play with it. >> >> There might also be room to factor out common functionality. We've >> also run into the need to print whole-module instead of containing >> construct (i.e. this loop). If we added upstream support for >> something along the lines of -print-module-after-all, building the >> git history could easily be done as a post processing step. >> >> Philip >> >> >> On 03/06/2018 10:43 AM, Alexandre Isoard via llvm-dev wrote: >>> Hello, >>> >>> I had a stupid idea recently that turned out not so stupid after >>> all. I wanted to be able to "see" an entire pass pipeline in action >>> to find unnecessary transformations and/or missed opportunities and >>> generally improve the debug-ability of LLVM. >>> >>> So as the title suggest, I implemented an equivalent of >>> "-print-after-all" but instead of printing into stdout I dump into a >>> file that get commit into a temporary git. There are some quirks >>> with it but it's working and is actually awesome. For example, at >>> first sight, I see multiple time lcssa and instcombine cancelling >>> each other's work. >>> >>> Of course, that has a big impact on compile time when enabled, but >>> that's still practical (git being quite good at its job) when debugging. >>> >>> There are improvement I can make, but would you guys be interested >>> in such feature? >>> >>> -- >>> *Alexandre Isoard* >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Fedor Sergeev via llvm-dev
2018-Mar-15 13:30 UTC
[llvm-dev] Commit module to Git after each Pass
On 03/15/2018 01:32 PM, Fedor Sergeev via llvm-dev wrote:> For this to be really usable in this setup we need additionally to: > - extend -print-module-scope to cover basic block passes > - introduce a clear way to separate module IRs as those are being > printed by -print-after-all > > But yes, it should work, and a wrapper that pipes to git fast-import > seems to be the best way to handle it.A simple 20-lines perl script does the trick pretty easily: https://pastebin.com/4J0b5Tr8 (this assumes my local modification to introduce the *** END OF ** IR DUMP marked at the end of -print-module-scope's IR module dump) ] git init ] RA/bin/opt -O3 some-ir.ll -disable-output -print-after-all -print-module-scope 2>&1 | filter-LLVM-ir-print.pl | git fast-import --done --date-format=now .... Majority of time is spent to actually print the IR (~2m for my testcase). Fast-import takes just a second. regards, Fedor.> > regards, > Fedor. > > On 03/15/2018 12:31 AM, Daniel Neilson via llvm-dev wrote: >> The print-module-after-all type of option exists in upstream: >> -print-module-scope - When >> printing IR for print-[before|after]{-all} always print a module IR >> >> commit 7d160f714357f6784ead669ce516e94991c12e5a >> Author: Fedor Sergeev <fedor.sergeev at azul.com >> <mailto:fedor.sergeev at azul.com>> >> Date: Fri Dec 1 17:42:46 2017 +0000 >> >> IR printing improvement for function passes - introducing >> -print-module-scope >> >> >> Summary: >> When debugging function passes it happens to be rather useful to >> dump >> the whole module before the transformation and then use this dump >> to analyze this single transformation by running it separately >> on that particular module state. >> >> >> Introducing >> -print-module-scope >> debugging option that forces all the function-level IR dumps >> to become whole-module dumps. >> >> >> This option builds on top of normal dumping controls like >> -print-before/after >> -filter-print-funcs >> >> >> The plan is to eventually extend this option to cover other local >> passes >> (at least loop passes) but that should go as a separate change. >> >> >> Loop passes here: >> commit 5608259c999fb77c5d6093895696f4daebe6b8cd >> Author: Fedor Sergeev <fedor.sergeev at azul.com >> <mailto:fedor.sergeev at azul.com>> >> Date: Fri Dec 1 18:33:58 2017 +0000 >> >> IR printing improvement for loop passes - handle -print-module-scope >> >> >> -Daniel >> >> >>> On Mar 14, 2018, at 3:51 PM, Philip Reames via llvm-dev >>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >>> >>> This is interesting, and might be useful. I don't know that this is >>> broadly useful enough for upstream inclusion, but if you could post >>> this to github somewhere, I might play with it. >>> >>> There might also be room to factor out common functionality. We've >>> also run into the need to print whole-module instead of containing >>> construct (i.e. this loop). If we added upstream support for >>> something along the lines of -print-module-after-all, building the >>> git history could easily be done as a post processing step. >>> >>> Philip >>> >>> >>> On 03/06/2018 10:43 AM, Alexandre Isoard via llvm-dev wrote: >>>> Hello, >>>> >>>> I had a stupid idea recently that turned out not so stupid after >>>> all. I wanted to be able to "see" an entire pass pipeline in action >>>> to find unnecessary transformations and/or missed opportunities and >>>> generally improve the debug-ability of LLVM. >>>> >>>> So as the title suggest, I implemented an equivalent of >>>> "-print-after-all" but instead of printing into stdout I dump into >>>> a file that get commit into a temporary git. There are some quirks >>>> with it but it's working and is actually awesome. For example, at >>>> first sight, I see multiple time lcssa and instcombine cancelling >>>> each other's work. >>>> >>>> Of course, that has a big impact on compile time when enabled, but >>>> that's still practical (git being quite good at its job) when >>>> debugging. >>>> >>>> There are improvement I can make, but would you guys be interested >>>> in such feature? >>>> >>>> -- >>>> *Alexandre Isoard* >>>> >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev