Displaying 20 results from an estimated 10000 matches similar to: "Commit module to Git after each Pass"
2018 Mar 14
0
Commit module to Git after each Pass
This is interesting, and might be useful. I don't know that this is
broadly useful enough for upstream inclusion, but if you could post this
to github somewhere, I might play with it.
There might also be room to factor out common functionality. We've also
run into the need to print whole-module instead of containing construct
(i.e. this loop). If we added upstream support for
2018 Mar 14
2
Commit module to Git after each Pass
The print-module-after-all type of option exists in upstream:
-print-module-scope - When printing IR for print-[before|after]{-all} always print a module IR
commit 7d160f714357f6784ead669ce516e94991c12e5a
Author: Fedor Sergeev <fedor.sergeev at azul.com<mailto:fedor.sergeev at azul.com>>
Date: Fri Dec 1 17:42:46 2017 +0000
IR
2018 Mar 15
2
Commit module to Git after each Pass
On 03/15/2018 01:32 PM, Fedor Sergeev via llvm-dev wrote:
> For this to be really usable in this setup we need additionally to:
> - extend -print-module-scope to cover basic block passes
> - introduce a clear way to separate module IRs as those are being
> printed by -print-after-all
>
> But yes, it should work, and a wrapper that pipes to git fast-import
> seems to be
2018 Mar 15
0
Commit module to Git after each Pass
For this to be really usable in this setup we need additionally to:
- extend -print-module-scope to cover basic block passes
- introduce a clear way to separate module IRs as those are being
printed by -print-after-all
But yes, it should work, and a wrapper that pipes to git fast-import
seems to be the best way to handle it.
regards,
Fedor.
On 03/15/2018 12:31 AM, Daniel Neilson via
2018 Mar 15
0
Commit module to Git after each Pass
Hmm...
I tried Alexandre's fix from D44244 and surprisingly it appears that
just using -print-module-scope w/o
any additional git actions is waaaay slower on my testcase than
-git-commit-module-all.
Hell, even a plan -print-after-all is slower:
] time R/bin/opt -O3 some-ir.ll -disable-output -git-commit-after-all
2>/dev/null
real 0m8.041s
user 0m7.133s
sys 0m0.936s
] time
2018 Mar 15
2
Commit module to Git after each Pass
Does git-commit-after-all print correctly after all the passes? Maybe I
messed it up and it skip some passes, therefore having less to do?
Either that, or piping has a higher cost than writing to file. Looks like
it surprisingly spends much less time in system more when going through
file. Maybe that's because the file is consistently around the same size
and is mmapped into memory
2018 Mar 15
3
Commit module to Git after each Pass
Does https://reviews.llvm.org/D44132 <https://reviews.llvm.org/D44132> help at all?
> On 15 Mar 2018, at 09:16, Philip Reames via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>
> The most likely answer is that the printer used by print-after-all is slow. I know there were some changes made around passing in some form of state cache (metadata related?) and that running
2018 Mar 15
4
Commit module to Git after each Pass
git-commit-after-all solution has one serious issue - it has a hardcoded
git handling which
makes it look problematic from many angles (picking a proper git,
selecting exact way of storing information, creating repository,
replacing the file etc etc).
Just dumping information in a way that allows easy subsequent machine
processing
seems to be a more flexible, less cluttered and overall clean
2018 Mar 15
2
Commit module to Git after each Pass
Huh. Great! 😁
I don't believe my poor excuse from earlier (else we should map all pipes
into files!), but I'm curious why we spend less time in system mode when
going through file than pipe. Maybe /dev/null is not as efficient as we
might think? I can't believe I'm saying that...
On Thu, Mar 15, 2018, 08:25 Fedor Sergeev <fedor.sergeev at azul.com> wrote:
> Well, git by
2018 Mar 15
0
Commit module to Git after each Pass
If this is faster than -print-after-all we may actually consider pushing
that in the code base then? (after diligent code review of course)
Note that it uses the same printing method as -print-after-all:
- create a pass of the same pass kind as the pass we just ran
- use Module::print(raw_ostream) to print (except -print-after-all only
print the concerned part and into stdout)
If there is
2018 Jun 14
3
Commit module to Git after each Pass
Hello,
Just an update on that. I am personally using -git-commit-after-all *as-is*
extremely frequently (combined with "git filter-branch" and "opt -S
-instnamer" it is extremely useful).
I unfortunately won't have time to write a better implementation of that,
and I agree "git fast-import" seems the way to go. If anybody is motivated
enough to do so, feel free.
2018 Mar 21
0
Commit module to Git after each Pass
On 03/16/2018 01:21 AM, Fedor Sergeev via llvm-dev wrote:
> git-commit-after-all solution has one serious issue - it has a
hardcoded git handling which
> makes it look problematic from many angles (picking a proper git,
> selecting exact way of storing information, creating repository,
replacing the file etc etc).
>
> Just dumping information in a way that allows easy
2018 Mar 15
0
Commit module to Git after each Pass
On 03/15/2018 06:09 PM, Alexandre Isoard wrote:
> Does git-commit-after-all print correctly after all the passes? Maybe
> I messed it up and it skip some passes, therefore having less to do?
I did verify that total amount of lines committed to git is reasonably high:
] git rev-list master | while read cmt; do git show $cmt:some-ir.ll;
done | wc -l
1587532
corresponding number for
2018 Mar 22
2
Commit module to Git after each Pass
Oh, well... as usually the answer appears to be pretty obvious.
99% of the time is spent inside the plain write.
-print-after-all prints into llvm::errs(), which is an *unbuffered*
raw_fd_stream.
And -git-commit-after-all opens a *buffered* raw_fd_stream.
As soon as I hacked -print-after-all to use a buffered stream to stderr
performance went
up to the normal expected values:
] time bin/opt
2018 Mar 21
1
Commit module to Git after each Pass
Do you really need to write the entire module to a single file? (Hence my
earlier hint...)
Why not write out a separate file for each def, so you don't need to dump
functions that haven't changed?
On Wed, Mar 21, 2018 at 8:38 PM, Fedor Sergeev via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 03/16/2018 01:21 AM, Fedor Sergeev via llvm-dev wrote:
> >
2018 Mar 15
0
Commit module to Git after each Pass
The most likely answer is that the printer used by print-after-all is
slow. I know there were some changes made around passing in some form
of state cache (metadata related?) and that running printers without
doing so work, but are dog slow. I suspect the print-after-all support
was never updated. Look at what we do for the normal IR emission "-S"
and see if print-after-all is
2018 Mar 07
2
Commit module to Git after each Pass
Today it is entirely in llvm. It is even more costly than -print-after-all
as it:
- print to file
- print the entire module (after each basic block, for basic block
passes, it will still print the entire module where only one basic block
changed)
- call git 2 times (add then commit) and wait for them to finish (I even
save all the empty commits)
The reason I print the entire module is so
2018 Mar 07
0
Commit module to Git after each Pass
On 6 March 2018 at 18:43, Alexandre Isoard via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> I had a stupid idea recently that turned out not so stupid after all. I
> wanted to be able to "see" an entire pass pipeline in action to find
> unnecessary transformations and/or missed opportunities and generally
> improve the debug-ability of LLVM.
>
>
2018 Mar 08
1
Commit module to Git after each Pass
Maybe. I can't compute the diff/patch for git though, it will have to do it
itself. Would that still work?
I am not sure I will have time to work on improving that.
For those who want to try it out I put it there:
https://reviews.llvm.org/D44244
And yes, this uses all possible forms of ugly. :-)
On Wed, Mar 7, 2018, 18:14 Jeremy Lakeman <Jeremy.Lakeman at gmail.com> wrote:
> Could
2018 Mar 08
0
Commit module to Git after each Pass
Could you format the output so that it is compatible with git fast-import?
https://www.git-scm.com/docs/git-fast-import
On Thu, Mar 8, 2018 at 2:34 AM, Alexandre Isoard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Today it is entirely in llvm. It is even more costly than -print-after-all
> as it:
> - print to file
> - print the entire module (after each basic block,