llvm dev - Sep 2013 - [LLVMdev] [RFC] Internal command line options should not be statically initialized.

On Sep 17, 2013, at 10:31 AM, Eric Christopher <echristo at gmail.com>
wrote:
> Hi Andy,
> 
> I definitely agree with the desire to remove command line options and
> having them be initialized as part of the pass would be general
> goodness. However, a few possible issues:
> 
> a) a number of command line options aren't really connected to passes
> per-se (backend options)
We don’t have to ban the old-style options. They can live-on indefinitely for
experimental purposes, but if test cases need to use those options, they would
need a REQUIRES: asserts line.

For options that we actually want to make available to tools (and general
testing) we have a couple possibilities:

Create a hook that can be called before command line parsing, like
initializeXXPass.

Raise those options into an API and define the command line interface at the
tool level instead. This seems like something we want to do anyway, but I’m not
proposing anything definite yet (I’m happy as long we’re moving in that
direction). We may want to raise some options all the way to function
attributes.
> b) "As long as pass initialization happens before parseCommandLine,
> usage should be consistent." I'm thinking this isn't going to
work for
> the opt tool at least :)
> 
> Thoughts?
I think all the standard passes will be initialized up front. The problem would
be with plugins using -load, but I fail to see how that can work with -help
today anyway.

-Andy
> 
> -eric
> 
> 
> On Tue, Sep 17, 2013 at 10:10 AM, Andrew Trick <atrick at apple.com>
wrote:
>> LLVM's internal command line library needs to evolve. We have an
immediate need to build LLVM as a library free of static initializers, but
before brute-force fixing this problem, I'd like outline the incremental
steps that will lead to a desirable long term solution. We want infrastructure
in place to provide an evolutionary path.
>> 
>> In the near term, clients who need llvm-the-library with no static
initializers will build with LLVM_NO_STATICINIT. In this mode, existing users of
cl::opt will default to being optimized away as constant initializers, and those
options will not be available for command line parsing.
>> 
>> A new option class will need be defined, cl::toolopt. Initially, this
will share the implementation and API with cl::opt. The only difference will be
that cl::toolopt is immune to LLVM_NO_STATICINIT. Options that are required for
tool support can simply be type-renamed to toolopt. Since these are not defined
in a library, their static initializers are irrelevant.
>> 
>> Eventually, we would like to eliminate the special LLVM_NO_STATICINIT
build mode. This can be done by making the -Asserts build just as strict. In the
meantime, we would like to run as many unit tests as possible with
LLVM_NO_STATICINIT builds. This will be solved by gradually moving cl::opt
definitions buried within LLVM libraries to to a new pattern that avoids static
initialization.
>> 
>> One easy pattern to follow is to register the option during pass
initialization with all the convenient flags and parameters, but refer to a
globally defined option storage that enforces the singleton and provides
visibility. As long as pass initialization happens before parseCommandLine,
usage should be consistent.
>> 
>> Strawman:
>> 
>> cl::optval<bool> MyOption; // Just the storage, no
initialization.
>> 
>> MyPass() {
>>  // Only registers an option with the same optval once.
>>  Option cl::registerOpt(MyOption, cl::init(false), cl::Hidden,
>>                         cl::desc("Descriptive string..."), );
>> }
>> 
>> -Andy
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

>> I definitely agree with the desire to remove command line options and
>> having them be initialized as part of the pass would be general
>> goodness. However, a few possible issues:
>>
>> a) a number of command line options aren't really connected to
passes
>> per-se (backend options)
>
> We don’t have to ban the old-style options. They can live-on indefinitely
for experimental purposes, but if test cases need to use those options, they
would need a REQUIRES: asserts line.
Enh, I'm in favor of banning them. Even though I use a few.
>
> For options that we actually want to make available to tools (and general
testing) we have a couple possibilities:
>
> Create a hook that can be called before command line parsing, like
initializeXXPass.
>
This seems meh as just a workaround for the current behavior. It'd
probably be a step forward, but might be more work than just doing...
> Raise those options into an API and define the command line interface at
the tool level instead. This seems like something we want to do anyway, but I’m
not proposing anything definite yet (I’m happy as long we’re moving in that
direction). We may want to raise some options all the way to function
attributes.
>
something like this. I think this is the way to go. I'm not sure how
it would look other than passing option structs into a context, but
that might work?
>> b) "As long as pass initialization happens before
parseCommandLine,
>> usage should be consistent." I'm thinking this isn't going
to work for
>> the opt tool at least :)
>>
>> Thoughts?
>
> I think all the standard passes will be initialized up front. The problem
would be with plugins using -load, but I fail to see how that can work with
-help today anyway.
>
Guess so. Seems weird, especially for slightly buggy cases like debug
info where most things happen during initialization. (Unfortunate, but
unraveling it is a pretty big project at the moment)

-eric
> -Andy
>
>>
>> -eric
>>
>>
>> On Tue, Sep 17, 2013 at 10:10 AM, Andrew Trick <atrick at
apple.com> wrote:
>>> LLVM's internal command line library needs to evolve. We have
an immediate need to build LLVM as a library free of static initializers, but
before brute-force fixing this problem, I'd like outline the incremental
steps that will lead to a desirable long term solution. We want infrastructure
in place to provide an evolutionary path.
>>>
>>> In the near term, clients who need llvm-the-library with no static
initializers will build with LLVM_NO_STATICINIT. In this mode, existing users of
cl::opt will default to being optimized away as constant initializers, and those
options will not be available for command line parsing.
>>>
>>> A new option class will need be defined, cl::toolopt. Initially,
this will share the implementation and API with cl::opt. The only difference
will be that cl::toolopt is immune to LLVM_NO_STATICINIT. Options that are
required for tool support can simply be type-renamed to toolopt. Since these are
not defined in a library, their static initializers are irrelevant.
>>>
>>> Eventually, we would like to eliminate the special
LLVM_NO_STATICINIT build mode. This can be done by making the -Asserts build
just as strict. In the meantime, we would like to run as many unit tests as
possible with LLVM_NO_STATICINIT builds. This will be solved by gradually moving
cl::opt definitions buried within LLVM libraries to to a new pattern that avoids
static initialization.
>>>
>>> One easy pattern to follow is to register the option during pass
initialization with all the convenient flags and parameters, but refer to a
globally defined option storage that enforces the singleton and provides
visibility. As long as pass initialization happens before parseCommandLine,
usage should be consistent.
>>>
>>> Strawman:
>>>
>>> cl::optval<bool> MyOption; // Just the storage, no
initialization.
>>>
>>> MyPass() {
>>>  // Only registers an option with the same optval once.
>>>  Option cl::registerOpt(MyOption, cl::init(false), cl::Hidden,
>>>                         cl::desc("Descriptive
string..."), );
>>> }
>>>
>>> -Andy
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

My understanding to this is limited. You can ignore this safely.

Can you introduce "before-all" and "after-all" passes that
does nothing more than handling flags and setting program statuses? That would
eliminate out-of-pass flags.

Sent from my iPhone

On 2013年9月18日, at 2:01, Eric Christopher <echristo at gmail.com> wrote:
>>> I definitely agree with the desire to remove command line options
and
>>> having them be initialized as part of the pass would be general
>>> goodness. However, a few possible issues:
>>> 
>>> a) a number of command line options aren't really connected to
passes
>>> per-se (backend options)
>> 
>> We don’t have to ban the old-style options. They can live-on
indefinitely for experimental purposes, but if test cases need to use those
options, they would need a REQUIRES: asserts line.
> 
> Enh, I'm in favor of banning them. Even though I use a few.
> 
>> 
>> For options that we actually want to make available to tools (and
general testing) we have a couple possibilities:
>> 
>> Create a hook that can be called before command line parsing, like
initializeXXPass.
> 
> This seems meh as just a workaround for the current behavior. It'd
> probably be a step forward, but might be more work than just doing...
> 
>> Raise those options into an API and define the command line interface
at the tool level instead. This seems like something we want to do anyway, but
I’m not proposing anything definite yet (I’m happy as long we’re moving in that
direction). We may want to raise some options all the way to function
attributes.
> 
> something like this. I think this is the way to go. I'm not sure how
> it would look other than passing option structs into a context, but
> that might work?
> 
>>> b) "As long as pass initialization happens before
parseCommandLine,
>>> usage should be consistent." I'm thinking this isn't
going to work for
>>> the opt tool at least :)
>>> 
>>> Thoughts?
>> 
>> I think all the standard passes will be initialized up front. The
problem would be with plugins using -load, but I fail to see how that can work
with -help today anyway.
> 
> Guess so. Seems weird, especially for slightly buggy cases like debug
> info where most things happen during initialization. (Unfortunate, but
> unraveling it is a pretty big project at the moment)
> 
> -eric
> 
>> -Andy
>> 
>>> 
>>> -eric
>>> 
>>> 
>>>> On Tue, Sep 17, 2013 at 10:10 AM, Andrew Trick <atrick at
apple.com> wrote:
>>>> LLVM's internal command line library needs to evolve. We
have an immediate need to build LLVM as a library free of static initializers,
but before brute-force fixing this problem, I'd like outline the incremental
steps that will lead to a desirable long term solution. We want infrastructure
in place to provide an evolutionary path.
>>>> 
>>>> In the near term, clients who need llvm-the-library with no
static initializers will build with LLVM_NO_STATICINIT. In this mode, existing
users of cl::opt will default to being optimized away as constant initializers,
and those options will not be available for command line parsing.
>>>> 
>>>> A new option class will need be defined, cl::toolopt.
Initially, this will share the implementation and API with cl::opt. The only
difference will be that cl::toolopt is immune to LLVM_NO_STATICINIT. Options
that are required for tool support can simply be type-renamed to toolopt. Since
these are not defined in a library, their static initializers are irrelevant.
>>>> 
>>>> Eventually, we would like to eliminate the special
LLVM_NO_STATICINIT build mode. This can be done by making the -Asserts build
just as strict. In the meantime, we would like to run as many unit tests as
possible with LLVM_NO_STATICINIT builds. This will be solved by gradually moving
cl::opt definitions buried within LLVM libraries to to a new pattern that avoids
static initialization.
>>>> 
>>>> One easy pattern to follow is to register the option during
pass initialization with all the convenient flags and parameters, but refer to a
globally defined option storage that enforces the singleton and provides
visibility. As long as pass initialization happens before parseCommandLine,
usage should be consistent.
>>>> 
>>>> Strawman:
>>>> 
>>>> cl::optval<bool> MyOption; // Just the storage, no
initialization.
>>>> 
>>>> MyPass() {
>>>> // Only registers an option with the same optval once.
>>>> Option cl::registerOpt(MyOption, cl::init(false), cl::Hidden,
>>>>                        cl::desc("Descriptive
string..."), );
>>>> }
>>>> 
>>>> -Andy
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On Sep 17, 2013, at 11:01 AM, Eric Christopher <echristo at gmail.com>
wrote:
>>> I definitely agree with the desire to remove command line options
and
>>> having them be initialized as part of the pass would be general
>>> goodness. However, a few possible issues:
>>> 
>>> a) a number of command line options aren't really connected to
passes
>>> per-se (backend options)
>> 
>> We don’t have to ban the old-style options. They can live-on
indefinitely for experimental purposes, but if test cases need to use those
options, they would need a REQUIRES: asserts line.
> 
> Enh, I'm in favor of banning them. Even though I use a few.
Sure. They’re convenient though. I’d rather have temporary experimental options
then no test cases at all for some feature during its development.
>> For options that we actually want to make available to tools (and
general testing) we have a couple possibilities:
>> 
>> Create a hook that can be called before command line parsing, like
initializeXXPass.
>> 
> 
> This seems meh as just a workaround for the current behavior. It'd
> probably be a step forward, but might be more work than just doing…
Fair enough. It’s a way to make options available for unit tests in -Asserts
builds without being forced to declare the option in the tool API. The most
attractive thing about this approach is that most experimental backend options
could be easily converted. So even if it is just a workaround, it provides a lot
of flexibility.
>> Raise those options into an API and define the command line interface
at the tool level instead. This seems like something we want to do anyway, but
I’m not proposing anything definite yet (I’m happy as long we’re moving in that
direction). We may want to raise some options all the way to function
attributes.
>> 
> 
> something like this. I think this is the way to go. I'm not sure how
> it would look other than passing option structs into a context, but
> that might work?
Ok, we all seem to want this. As I told Hal, we probably should have a
declarative syntax that generates code for tool options, integrates function
attributes and command line flags, and provides an central lookup with some
reflection API. How we do this us not going to change how we solve the immediate
problem of static initializers, and it will obviously take a lot of time to
agree on the design. So I’ll declare it out of scope for now.

-Andy
>>> b) "As long as pass initialization happens before
parseCommandLine,
>>> usage should be consistent." I'm thinking this isn't
going to work for
>>> the opt tool at least :)
>>> 
>>> Thoughts?
>> 
>> I think all the standard passes will be initialized up front. The
problem would be with plugins using -load, but I fail to see how that can work
with -help today anyway.
>> 
> 
> Guess so. Seems weird, especially for slightly buggy cases like debug
> info where most things happen during initialization. (Unfortunate, but
> unraveling it is a pretty big project at the moment)
> 
> -eric
> 
>> -Andy
>> 
>>> 
>>> -eric
>>> 
>>> 
>>> On Tue, Sep 17, 2013 at 10:10 AM, Andrew Trick <atrick at
apple.com> wrote:
>>>> LLVM's internal command line library needs to evolve. We
have an immediate need to build LLVM as a library free of static initializers,
but before brute-force fixing this problem, I'd like outline the incremental
steps that will lead to a desirable long term solution. We want infrastructure
in place to provide an evolutionary path.
>>>> 
>>>> In the near term, clients who need llvm-the-library with no
static initializers will build with LLVM_NO_STATICINIT. In this mode, existing
users of cl::opt will default to being optimized away as constant initializers,
and those options will not be available for command line parsing.
>>>> 
>>>> A new option class will need be defined, cl::toolopt.
Initially, this will share the implementation and API with cl::opt. The only
difference will be that cl::toolopt is immune to LLVM_NO_STATICINIT. Options
that are required for tool support can simply be type-renamed to toolopt. Since
these are not defined in a library, their static initializers are irrelevant.
>>>> 
>>>> Eventually, we would like to eliminate the special
LLVM_NO_STATICINIT build mode. This can be done by making the -Asserts build
just as strict. In the meantime, we would like to run as many unit tests as
possible with LLVM_NO_STATICINIT builds. This will be solved by gradually moving
cl::opt definitions buried within LLVM libraries to to a new pattern that avoids
static initialization.
>>>> 
>>>> One easy pattern to follow is to register the option during
pass initialization with all the convenient flags and parameters, but refer to a
globally defined option storage that enforces the singleton and provides
visibility. As long as pass initialization happens before parseCommandLine,
usage should be consistent.
>>>> 
>>>> Strawman:
>>>> 
>>>> cl::optval<bool> MyOption; // Just the storage, no
initialization.
>>>> 
>>>> MyPass() {
>>>> // Only registers an option with the same optval once.
>>>> Option cl::registerOpt(MyOption, cl::init(false), cl::Hidden,
>>>>                        cl::desc("Descriptive
string..."), );
>>>> }
>>>> 
>>>> -Andy
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>