llvm dev - Oct 2010 - [LLVMdev] MAJOR API CHANGE: Pass initialization without static constructors

On Oct 8, 2010, at 1:05 PM, John Criswell wrote:
> On 10/8/10 1:29 PM, Owen Anderson wrote:
>> Hello fellow LLVM-ers,
>> 
>> As those of you who follow llvm-commits have probably noticed, I've
been hard at work reworking our pass infrastructure to perform pass
initialization/registration without static constructors.  This is a boon for
clients of the LLVM libraries, as they can now control when and if
initialization occurs, as opposed to being forced to incur the initialization
cost at startup, negatively impacting their startup times.
>> 
>> The new-style solution consists of an llvm::initializeMyPass method for
every pass.
> 
> What is this method of our passes supposed to do?  Is there a default
implementation in the Pass class?
This method is provided for you by INITIALIZE_PASS macros (and friends).  All
you have to do is add the declaration to InitializePasses.h, and add a call of
it to the library's batch initialization method.
>>   These initialization routines must be invoked before the PassManager
will be able to resolve pipelines involving your pass.  For convenience, LLVM
now exposes per-library batch initialization routines, i.e.
initializeScalarOpts(), which initialize all the passes within a given library
in a single go.
> 
> From an ease of programming perspective, this solution is not ideal as what
we had before.  Each time I link in a new library into my program, I have to go
find the method that initializes all of the passes and remember to call it from
my tool.  Could you devise something better?  For example, could you extend
PassManager with a method that calls all of the passes's initialization
methods?  In other words, could you easily support something like:
> 
> PassManager.add (Pass1);
> PassManager.add (Pass2);
> ...
> PassManager.add (PassN);
> PassManager.initializeAllPasses ();
> PassManager.run();
> 
> 
> ... or could the PassManager::add() method automatically initialize the
pass?  Other possibilities may exists which are better as well; these are just
the first two I've thought of.
Neither of those will work.  These initialization have nothing to do with
initializing the pass, per se, and everything to do with the PassManager being
able to look it up.  In order for the PassManager to be able to find a given
pass (say, because some pass you explicitly requested depends on it), that
pass's corresponding PassInfo MUST have been registered beforehand with the
PassManager, enabling it to perform recursive dependency chasing.  Without that
pre-registration, it has no way of knowing what passes exist, and is thus unable
to resolve dependencies.

At best, such an approach would allow you to avoid explicit initialization of
the passes you are explicitly instantiating, but anything that must be found by
dependency resolution needs to be registered with the PassRegistry beforehand.

--Owen

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20101008/c5b3d2bc/attachment.html>

On 10/8/10 3:15 PM, Owen Anderson wrote:
>
> On Oct 8, 2010, at 1:05 PM, John Criswell wrote:
>
>> On 10/8/10 1:29 PM, Owen Anderson wrote:
>>> Hello fellow LLVM-ers,
>>>
>>> As those of you who follow llvm-commits have probably noticed,
I've
>>> been hard at work reworking our pass infrastructure to perform pass
>>> initialization/registration without static constructors.  This is a
>>> boon for clients of the LLVM libraries, as they can now control
when
>>> and if initialization occurs, as opposed to being forced to incur 
>>> the initialization cost at startup, negatively impacting their 
>>> startup times.
>>>
>>> The new-style solution consists of an llvm::initializeMyPass method
>>> for every pass.
>>
>> What is this method of our passes supposed to do?  Is there a default 
>> implementation in the Pass class?
>
> This method is provided for you by INITIALIZE_PASS macros (and 
> friends).  All you have to do is add the declaration to 
> InitializePasses.h, and add a call of it to the library's batch 
> initialization method.
Is InitializePasses.h an LLVM header file?  If so, I assume modifying 
InitializePasses.h is only required if the pass is part of LLVM.  
Out-of-tree projects need to be able to create new passes without 
modifying the LLVM source.

>>>   These initialization routines must be invoked before the 
>>> PassManager will be able to resolve pipelines involving your pass. 
>>>  For convenience, LLVM now exposes per-library batch initialization
>>> routines, i.e. initializeScalarOpts(), which initialize all the 
>>> passes within a given library in a single go.
>>
>> From an ease of programming perspective, this solution is not ideal 
>> as what we had before.  Each time I link in a new library into my 
>> program, I have to go find the method that initializes all of the 
>> passes and remember to call it from my tool.  Could you devise 
>> something better?  For example, could you extend PassManager with a 
>> method that calls all of the passes's initialization methods?  In 
>> other words, could you easily support something like:
>>
>> PassManager.add (Pass1);
>> PassManager.add (Pass2);
>> ...
>> PassManager.add (PassN);
>> PassManager.initializeAllPasses ();
>> PassManager.run();
>>
>>
>> ... or could the PassManager::add() method automatically initialize 
>> the pass?  Other possibilities may exists which are better as well; 
>> these are just the first two I've thought of.
>
> Neither of those will work.  These initialization have nothing to do 
> with initializing the pass, per se, and everything to do with the 
> PassManager being able to look it up.  In order for the PassManager to 
> be able to find a given pass (say, because some pass you explicitly 
> requested depends on it), that pass's corresponding PassInfo MUST have 
> been registered beforehand with the PassManager, enabling it to 
> perform recursive dependency chasing.  Without that pre-registration, 
> it has no way of knowing what passes exist, and is thus unable to 
> resolve dependencies.
Hrm.  I see.

I still don't like the idea of having every statically-linked tool 
explicitly initializing every library that gets linked in.  Just dumping 
the library into the Makefile and being done with it was much nicer.

If you can find a reasonable way to support that, it would be nice.  
However, if you can't, it's not that big a deal.  As I mentioned before,
as long as out-of-tree passes don't have to modify LLVM source files to 
work properly, I'll live.

-- John T.
>
> At best, such an approach would allow you to avoid explicit 
> initialization of the passes you are explicitly instantiating, but 
> anything that must be found by dependency resolution needs to be 
> registered with the PassRegistry beforehand.
>
> --Owen
>

On Oct 8, 2010, at 1:37 PM, John Criswell wrote:
> On 10/8/10 3:15 PM, Owen Anderson wrote:
>> 
>> On Oct 8, 2010, at 1:05 PM, John Criswell wrote:
>> 
>>> On 10/8/10 1:29 PM, Owen Anderson wrote:
>>>> Hello fellow LLVM-ers,
>>>> 
>>>> As those of you who follow llvm-commits have probably noticed,
I've been hard at work reworking our pass infrastructure to perform pass
initialization/registration without static constructors.  This is a boon for
clients of the LLVM libraries, as they can now control when and if
initialization occurs, as opposed to being forced to incur the initialization
cost at startup, negatively impacting their startup times.
>>>> 
>>>> The new-style solution consists of an llvm::initializeMyPass
method for every pass.
>>> 
>>> What is this method of our passes supposed to do?  Is there a
default implementation in the Pass class?
>> 
>> This method is provided for you by INITIALIZE_PASS macros (and
friends).  All you have to do is add the declaration to InitializePasses.h, and
add a call of it to the library's batch initialization method.
> 
> Is InitializePasses.h an LLVM header file?  If so, I assume modifying
InitializePasses.h is only required if the pass is part of LLVM.  Out-of-tree
projects need to be able to create new passes without modifying the LLVM source.
I was answering within the context of implementing a new pass in LLVM, where
it's an API requirement that pass initializations are exposed in
InitializePasses.h.  Within your own tool, it doesn't really matter where
you declare it as long as it's visible to wherever you call it from. :-)

Also, as I pointed out, the RegisterPass<> templates do continue to exist
and function without the need for explicit initialization(and are necessary for
continued support of dynamically loadable plugins), but come with the cost of a
static constructor.  For a pass wholly contained within an out-of-tree tool, it
would be perfectly reasonable to use RegisterPass<> if the cost is
acceptable to you.
> Hrm.  I see.
> 
> I still don't like the idea of having every statically-linked tool
explicitly initializing every library that gets linked in.  Just dumping the
library into the Makefile and being done with it was much nicer.
> 
> If you can find a reasonable way to support that, it would be nice. 
However, if you can't, it's not that big a deal.  As I mentioned before,
as long as out-of-tree passes don't have to modify LLVM source files to work
properly, I'll live.
I don't especially like it either, but the abundance of static constructors
in LLVM has been a long-standing performance concern.  I wouldn't be going
this way if I had a better solution.

--Owen