llvm dev - May 2015 - [LLVMdev] (Possibly buggy?) doFinalization method behavior of FunctionPass

Hello again,

First of all, thanks for all the answers =) they really helped a lot =D

*Have you verified that some other pass is not adding the function
declarations back in after your pass is executed (e.g., by using the
-debug-pass=Executions argument to see what passes run after your pass)?*

I considered that for a moment, but I realized that wouldn't be possible
for two reasons: I specifically ran my pass with opt level zero; and one of
the tasks I was doing was removing unused prototypes from the Module -- so,
why would any other doFinalization pass reintroduce those prototypes if
those functions were not being invoked anywhere? (PS: since this clean-up
happened inside my doFinalization, I'm assuming only other doFinalization
would be able to change the module after this point)


*Also, does your doFinalization() method make other changes that are
persistent?*


The main reason why I chose to use doFinalization was to check if any of
the functions I'm replacing still had any uses: since my pass replaced
every function call with my respective function calls, if any uses could
still be listed, that meant that some instruction must be taking the
address of that function, and I should treat those instructions at this
point. At the end, since I was already checking for uses, I ended up
removing the extra prototypes at this point, if they were still there.

I don’t think doFinalization() is really meant to be used this way.

Its purpose is to allow clean-up of internal data-structures used by the
pass itself, not to make additional changes to the module.


Then, I have a question: why should doFinalization need a Module& as
argument? I mean, since I shouldn't be able to modify it, and all data
structures I created to work my pass are known and (mostly) directly
accessible from within doFinalization, its argument would never be useful,
right?

One option would be to rewrite your pass as a ModulePass instead of a
FunctionPass, then iterating over the functions manually, and doing the
final clean-up once that’s done.

*That said, I agree with the suggestion of writing a ModulePass.  Since the
PassManager does not run FunctionPasses in parallel yet, there's little
benefit to using them.  I have often found the limitations on
FunctionPasses to not be worth the hassle.*

Well, I actually wouldn't mind using a ModulePass to begin with, but
unfortunately I had to rewrite my pass as a FunctionPass because part of
the job it does involve accessing the DominatorTree -- and, unfortunately,
I couldn't find a way to access it from inside a ModulePass. Even when I
try to force a dependence between my module pass and DominatorTree, I can't
get it to work because (if I'm not mistaken) module passes are hard-coded
to run before function passes =/

*My understanding is that doInitialization() and doFinalization() are
designed specifically for modifying the LLVM IR (otherwise, why would a
mutable reference to the Function be provided)?*


*If that is not the case, then there is either a bug in the code or a bug
in the documentation.*

Exactly like I thought =)


I’ve always thought that the only guarantee is that doFinalization(Module
&M) runs after runOnFunction() was executed for all functions in M, and
there’s no guarantee it runs *immediately* after.

That is, a PM may run a bunch of function passes over each function, and
only then call doFinazliation() for each pass. That means that, even though
you get a mutable reference to the module, the module you’ll see is quite
different from what you may expect.



People more familiar with the pass managers – please correct me if I’m
wrong.


I don't really mind at all if other passes run or even had their
doFinalization passes running in any order in relation to mine: the point
here is doFinalization is running (I know that because module.dump() is
being called from inside that function, and I can see the module containing
the modifications I do during the execution of that method), but the final
bc file (written by opt) does have the exactly same code that I get if I
dumped module at the beginning of my doFinalization (and they have
differences between them btw =)).




--
Cristianno Martins
PhD Student of Computer Science
University of Campinas
cmartins at ic.unicamp.br
<cristiannomartins at hotmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150506/2cb31db5/attachment.html>

Btw, I forgot to add this piece of information: I ended up dividing the
work of my pass into two parts -- I created a module pass that had all the
transformation instructions that didn't have anything to do with dominance,
and I kept the dominance-dependent code inside a function pass. Then, I
just had to force a dependence between them (the function pass requiring
the module one), and this way I could apply my transformations.

Anyways, I still think doFinalization is wrongly defined or, at least,
implemented, as I tried to discuss on my previous email =)

Thanks a lot,


--
Cristianno Martins
PhD Student of Computer Science
University of Campinas
cmartins at ic.unicamp.br
<cristiannomartins at hotmail.com>

On Wed, May 6, 2015 at 1:11 PM, Cristianno Martins <
cristiannomartins at gmail.com> wrote:
> Hello again,
>
> First of all, thanks for all the answers =) they really helped a lot =D
>
> *Have you verified that some other pass is not adding the function
> declarations back in after your pass is executed (e.g., by using the
> -debug-pass=Executions argument to see what passes run after your pass)?*
>
> I considered that for a moment, but I realized that wouldn't be
possible
> for two reasons: I specifically ran my pass with opt level zero; and one of
> the tasks I was doing was removing unused prototypes from the Module -- so,
> why would any other doFinalization pass reintroduce those prototypes if
> those functions were not being invoked anywhere? (PS: since this clean-up
> happened inside my doFinalization, I'm assuming only other
doFinalization
> would be able to change the module after this point)
>
>
> *Also, does your doFinalization() method make other changes that are
> persistent?*
>
>
> The main reason why I chose to use doFinalization was to check if any of
> the functions I'm replacing still had any uses: since my pass replaced
> every function call with my respective function calls, if any uses could
> still be listed, that meant that some instruction must be taking the
> address of that function, and I should treat those instructions at this
> point. At the end, since I was already checking for uses, I ended up
> removing the extra prototypes at this point, if they were still there.
>
> I don’t think doFinalization() is really meant to be used this way.
>
> Its purpose is to allow clean-up of internal data-structures used by the
> pass itself, not to make additional changes to the module.
>
>
> Then, I have a question: why should doFinalization need a Module& as
> argument? I mean, since I shouldn't be able to modify it, and all data
> structures I created to work my pass are known and (mostly) directly
> accessible from within doFinalization, its argument would never be useful,
> right?
>
> One option would be to rewrite your pass as a ModulePass instead of a
> FunctionPass, then iterating over the functions manually, and doing the
> final clean-up once that’s done.
>
> *That said, I agree with the suggestion of writing a ModulePass.  Since
> the PassManager does not run FunctionPasses in parallel yet, there's
little
> benefit to using them.  I have often found the limitations on
> FunctionPasses to not be worth the hassle.*
>
> Well, I actually wouldn't mind using a ModulePass to begin with, but
> unfortunately I had to rewrite my pass as a FunctionPass because part of
> the job it does involve accessing the DominatorTree -- and, unfortunately,
> I couldn't find a way to access it from inside a ModulePass. Even when
I
> try to force a dependence between my module pass and DominatorTree, I
can't
> get it to work because (if I'm not mistaken) module passes are
hard-coded
> to run before function passes =/
>
> *My understanding is that doInitialization() and doFinalization() are
> designed specifically for modifying the LLVM IR (otherwise, why would a
> mutable reference to the Function be provided)?*
>
>
> *If that is not the case, then there is either a bug in the code or a bug
> in the documentation.*
>
> Exactly like I thought =)
>
>
> I’ve always thought that the only guarantee is that doFinalization(Module
> &M) runs after runOnFunction() was executed for all functions in M, and
> there’s no guarantee it runs *immediately* after.
>
> That is, a PM may run a bunch of function passes over each function, and
> only then call doFinazliation() for each pass. That means that, even though
> you get a mutable reference to the module, the module you’ll see is quite
> different from what you may expect.
>
>
>
> People more familiar with the pass managers – please correct me if I’m
> wrong.
>
>
> I don't really mind at all if other passes run or even had their
> doFinalization passes running in any order in relation to mine: the point
> here is doFinalization is running (I know that because module.dump() is
> being called from inside that function, and I can see the module containing
> the modifications I do during the execution of that method), but the final
> bc file (written by opt) does have the exactly same code that I get if I
> dumped module at the beginning of my doFinalization (and they have
> differences between them btw =)).
>
>
>
>
> --
> Cristianno Martins
> PhD Student of Computer Science
> University of Campinas
> cmartins at ic.unicamp.br
> <cristiannomartins at hotmail.com>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150506/6dba527a/attachment.html>

Only one little correction:

I don't really mind at all if other passes run or even had
their
> doFinalization passes running in any order in relation to mine: the point
> here is doFinalization is running (I know that because module.dump() is
> being called from inside that function, and I can see the module containing
> the modifications I do during the execution of that method), but the final
> bc file (written by opt) does have the exactly same code that I get if I
> dumped module *at the beginning* of my doFinalization (and they have
> differences between them btw =)).
>
Here, I meant to say "At the end" =)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150506/548e26b4/attachment.html>

I think it’s wrongly documented. :-)

As a side note, I believe requiring transformation passes (as opposed to
requiring analysis passes) is also not-quite-supported.

From: Cristianno Martins [mailto:cristiannomartins at gmail.com]
Sent: Wednesday, May 06, 2015 19:16
To: Kuperstein, Michael M
Cc: John Criswell; Lista LLVM-dev
Subject: Re: [LLVMdev] (Possibly buggy?) doFinalization method behavior of
FunctionPass

Btw, I forgot to add this piece of information: I ended up dividing the work of
my pass into two parts -- I created a module pass that had all the
transformation instructions that didn't have anything to do with dominance,
and I kept the dominance-dependent code inside a function pass. Then, I just had
to force a dependence between them (the function pass requiring the module one),
and this way I could apply my transformations.

Anyways, I still think doFinalization is wrongly defined or, at least,
implemented, as I tried to discuss on my previous email =)

Thanks a lot,


--
Cristianno Martins
PhD Student of Computer Science
University of Campinas
cmartins at ic.unicamp.br<mailto:cmartins at ic.unicamp.br>

On Wed, May 6, 2015 at 1:11 PM, Cristianno Martins <cristiannomartins at
gmail.com<mailto:cristiannomartins at gmail.com>> wrote:
Hello again,

First of all, thanks for all the answers =) they really helped a lot =D

Have you verified that some other pass is not adding the function declarations
back in after your pass is executed (e.g., by using the -debug-pass=Executions
argument to see what passes run after your pass)?

I considered that for a moment, but I realized that wouldn't be possible for
two reasons: I specifically ran my pass with opt level zero; and one of the
tasks I was doing was removing unused prototypes from the Module -- so, why
would any other doFinalization pass reintroduce those prototypes if those
functions were not being invoked anywhere? (PS: since this clean-up happened
inside my doFinalization, I'm assuming only other doFinalization would be
able to change the module after this point)

Also, does your doFinalization() method make other changes that are persistent?

The main reason why I chose to use doFinalization was to check if any of the
functions I'm replacing still had any uses: since my pass replaced every
function call with my respective function calls, if any uses could still be
listed, that meant that some instruction must be taking the address of that
function, and I should treat those instructions at this point. At the end, since
I was already checking for uses, I ended up removing the extra prototypes at
this point, if they were still there.

I don’t think doFinalization() is really meant to be used this way.
Its purpose is to allow clean-up of internal data-structures used by the pass
itself, not to make additional changes to the module.

Then, I have a question: why should doFinalization need a Module& as
argument? I mean, since I shouldn't be able to modify it, and all data
structures I created to work my pass are known and (mostly) directly accessible
from within doFinalization, its argument would never be useful, right?

One option would be to rewrite your pass as a ModulePass instead of a
FunctionPass, then iterating over the functions manually, and doing the final
clean-up once that’s done.

That said, I agree with the suggestion of writing a ModulePass.  Since the
PassManager does not run FunctionPasses in parallel yet, there's little
benefit to using them.  I have often found the limitations on FunctionPasses to
not be worth the hassle.

Well, I actually wouldn't mind using a ModulePass to begin with, but
unfortunately I had to rewrite my pass as a FunctionPass because part of the job
it does involve accessing the DominatorTree -- and, unfortunately, I
couldn't find a way to access it from inside a ModulePass. Even when I try
to force a dependence between my module pass and DominatorTree, I can't get
it to work because (if I'm not mistaken) module passes are hard-coded to run
before function passes =/

My understanding is that doInitialization() and doFinalization() are designed
specifically for modifying the LLVM IR (otherwise, why would a mutable reference
to the Function be provided)?

If that is not the case, then there is either a bug in the code or a bug in the
documentation.

Exactly like I thought =)

I’ve always thought that the only guarantee is that doFinalization(Module
&M) runs after runOnFunction() was executed for all functions in M, and
there’s no guarantee it runs *immediately* after.
That is, a PM may run a bunch of function passes over each function, and only
then call doFinazliation() for each pass. That means that, even though you get a
mutable reference to the module, the module you’ll see is quite different from
what you may expect.

People more familiar with the pass managers – please correct me if I’m wrong.

I don't really mind at all if other passes run or even had their
doFinalization passes running in any order in relation to mine: the point here
is doFinalization is running (I know that because module.dump() is being called
from inside that function, and I can see the module containing the modifications
I do during the execution of that method), but the final bc file (written by
opt) does have the exactly same code that I get if I dumped module at the
beginning of my doFinalization (and they have differences between them btw =)).



--
Cristianno Martins
PhD Student of Computer Science
University of Campinas
cmartins at ic.unicamp.br<mailto:cmartins at ic.unicamp.br>


---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150507/f272bcc0/attachment.html>