llvm dev - Dec 2016 - Enabling statistics in release builds / static constructors

> On Dec 13, 2016, at 12:56 PM, Reid Kleckner <rnk at google.com>
wrote:
> 
> Given that LLVM has so many auto-registration systems (cl::opt, target
registry, pass registry, statistics, I'm sure there's more), maybe we
should spend the time to build an auto-registration system that doesn't
involve static constructors?
I would volunteer to do the work, however this obviously needs some consensus
first on how that would look.
> 
> It needs custom code for every supported object file format, and is hard to
get right when DSOs are involved, but in the long run it's probably worth
fixing this problem once and for all.
I assume you are thinking about creating custom linker sections with list of
init functions; Similar to the existing constructors sections but running at a
time controlled by llvm code. While the compiler/linker nerd in me would love
doing that, I could see this being very tricky to pull off consistenly on all
platforms.

We should not forget that there is a portable and proven solution: Just write
the code!
So here comes the strawman:


static Statistic NumBlips("blips");
static cl::opt MyOpt("my-cool-option", cl::desc("bla"));
static cl::opt AnotherOpt("bla", cl::desc("foo bar"));
// Note that the constructors of Statistic and cl::opt would be reworked to be
pure constexpr and do not run any code

static void init_globals() {
  NumBlips.init();
  MyOpt.init();
  AnotherOpt.init();
}
// Note that the init_globals() function is pretty mechanical so hopefully easy
to understand and maintain.

We have to call init_gloabals somewhere early:
- Put an init_globals() call into code that runs early.
- We already have a lot of early running functions called initializeXXXPass()
which we can use for this
- For the remaining files we probably have to export init_globals() and call it
from some common place of the library.

If someone comes up a with working solution for putting initializers into a
section we can later replace the
init_globals() function with that so the refactoring work to split into
constexpr constructor and init() is not wasted either way!

- Matthias

On Tue, Dec 13, 2016 at 1:22 PM, Matthias Braun <mbraun at apple.com>
wrote:
>
> > On Dec 13, 2016, at 12:56 PM, Reid Kleckner <rnk at google.com>
wrote:
> >
> > Given that LLVM has so many auto-registration systems (cl::opt, target
> registry, pass registry, statistics, I'm sure there's more), maybe
we
> should spend the time to build an auto-registration system that doesn't
> involve static constructors?
> I would volunteer to do the work, however this obviously needs some
> consensus first on how that would look.
>
> >
> > It needs custom code for every supported object file format, and is
hard
> to get right when DSOs are involved, but in the long run it's probably
> worth fixing this problem once and for all.
> I assume you are thinking about creating custom linker sections with list
> of init functions; Similar to the existing constructors sections but
> running at a time controlled by llvm code. While the compiler/linker nerd
> in me would love doing that, I could see this being very tricky to pull off
> consistenly on all platforms.
>
The goal of using sections would be to build the array of statistics
without running any code. This is only possible when you have one DSO. Once
you have a second DSO, it will need to run some initialization code to link
the two arrays together.

We should not forget that there is a portable and proven solution:
Just
> write the code!

Yep, that always works, and you can control when it happens so you don't
have to run initializers scattered across your binary at startup.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161213/43873af1/attachment.html>

> On Dec 13, 2016, at 1:22 PM, Matthias Braun via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> 
>> On Dec 13, 2016, at 12:56 PM, Reid Kleckner <rnk at google.com>
wrote:
>> 
>> Given that LLVM has so many auto-registration systems (cl::opt, target
registry, pass registry, statistics, I'm sure there's more), maybe we
should spend the time to build an auto-registration system that doesn't
involve static constructors?
> I would volunteer to do the work, however this obviously needs some
consensus first on how that would look.
> 
>> 
>> It needs custom code for every supported object file format, and is
hard to get right when DSOs are involved, but in the long run it's probably
worth fixing this problem once and for all.
> I assume you are thinking about creating custom linker sections with list
of init functions; Similar to the existing constructors sections but running at
a time controlled by llvm code. While the compiler/linker nerd in me would love
doing that, I could see this being very tricky to pull off consistenly on all
platforms.
> 
> We should not forget that there is a portable and proven solution: Just
write the code!
I agree, and there was a proposal in the past for cl::opt (I believe from Chris
B.). The idea would be to have explicit registration (from the pass ctor for
instance) and the storage for the options being hold in the context somehow (old
memories, not sure about the details).

CC Chris who likely has more information (and possibly pointers).

— 
Mehdi


> So here comes the strawman:
> 
> 
> static Statistic NumBlips("blips");
> static cl::opt MyOpt("my-cool-option",
cl::desc("bla"));
> static cl::opt AnotherOpt("bla", cl::desc("foo bar"));
> // Note that the constructors of Statistic and cl::opt would be reworked to
be pure constexpr and do not run any code
> 
> static void init_globals() {
>  NumBlips.init();
>  MyOpt.init();
>  AnotherOpt.init();
> }
> // Note that the init_globals() function is pretty mechanical so hopefully
easy to understand and maintain.
> 
> We have to call init_gloabals somewhere early:
> - Put an init_globals() call into code that runs early.
> - We already have a lot of early running functions called
initializeXXXPass() which we can use for this
> - For the remaining files we probably have to export init_globals() and
call it from some common place of the library.
> 
> If someone comes up a with working solution for putting initializers into a
section we can later replace the
> init_globals() function with that so the refactoring work to split into
constexpr constructor and init() is not wasted either way!
> 
> - Matthias
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

> On Dec 13, 2016, at 3:23 PM, Mehdi Amini <mehdi.amini at apple.com>
wrote:
> 
>> 
>> On Dec 13, 2016, at 1:22 PM, Matthias Braun via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>> 
>> 
>>> On Dec 13, 2016, at 12:56 PM, Reid Kleckner <rnk at
google.com> wrote:
>>> 
>>> Given that LLVM has so many auto-registration systems (cl::opt,
target registry, pass registry, statistics, I'm sure there's more),
maybe we should spend the time to build an auto-registration system that
doesn't involve static constructors?
>> I would volunteer to do the work, however this obviously needs some
consensus first on how that would look.
>> 
>>> 
>>> It needs custom code for every supported object file format, and is
hard to get right when DSOs are involved, but in the long run it's probably
worth fixing this problem once and for all.
>> I assume you are thinking about creating custom linker sections with
list of init functions; Similar to the existing constructors sections but
running at a time controlled by llvm code. While the compiler/linker nerd in me
would love doing that, I could see this being very tricky to pull off
consistenly on all platforms.
>> 
>> We should not forget that there is a portable and proven solution: Just
write the code!
> 
> I agree, and there was a proposal in the past for cl::opt (I believe from
Chris B.). The idea would be to have explicit registration (from the pass ctor
for instance) and the storage for the options being hold in the context somehow
(old memories, not sure about the details).
> 
> CC Chris who likely has more information (and possibly pointers).
You probably mean:
http://web.archive.org/http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-August/075886.html
<http://web.archive.org/http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-August/075886.html>
[1]
and
http://reviews.llvm.org/D5389 <http://reviews.llvm.org/D5389>

but I fail to see how that helps Statistic variables which are still global.

- Matthias

[1] Please someone please give us an easier time to access old mails when you
just have the old link...

>  
> Mehdi
> 
> 
> 
>> So here comes the strawman:
>> 
>> 
>> static Statistic NumBlips("blips");
>> static cl::opt MyOpt("my-cool-option",
cl::desc("bla"));
>> static cl::opt AnotherOpt("bla", cl::desc("foo
bar"));
>> // Note that the constructors of Statistic and cl::opt would be
reworked to be pure constexpr and do not run any code
>> 
>> static void init_globals() {
>> NumBlips.init();
>> MyOpt.init();
>> AnotherOpt.init();
>> }
>> // Note that the init_globals() function is pretty mechanical so
hopefully easy to understand and maintain.
>> 
>> We have to call init_gloabals somewhere early:
>> - Put an init_globals() call into code that runs early.
>> - We already have a lot of early running functions called
initializeXXXPass() which we can use for this
>> - For the remaining files we probably have to export init_globals() and
call it from some common place of the library.
>> 
>> If someone comes up a with working solution for putting initializers
into a section we can later replace the
>> init_globals() function with that so the refactoring work to split into
constexpr constructor and init() is not wasted either way!
>> 
>> - Matthias
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161213/63cc8c5b/attachment.html>