llvm dev - Jan 2011 - [LLVMdev] Marking a function prototype as being "persistent"

On 6 janv. 2011, at 19:00, llvmdev-request at cs.uiuc.edu wrote:
>> I'm using LLVM to JIT-compile the XL programming language. I've
recently added a whole-program optimization phase that gives excellent results,
but I noticed that the StripDeadPrototypesPass was removing all references to my
runtime support functions.
> 
> why is that a problem?
Because I have pointers to functions which suddenly go bad.
>> Apparently, this depends on the oddly-named UnitAtATime parameter. Set
it to false. Try again. Suggestion: what about a comment explaining what
"UnitAtATime" means :-)
> 
> If UnitAtATime is false this means that you are optimizing functions as you
are
> generating them, rather than first generating all functions and other
globals
> and only then optimizing.  It's basically a historical anachronism
coming from
> the way GCC used to work.
Thanks for explaining.

Why not redefine it from historical anachronism to architectural flag, that
indicates you may generate more code with the same functions and globals later?
And then, you could pass the same flag to createStandardLTOPasses, and that
would skip the GlobalDCE passes.

>> Even then, my runtime function prorotypes are still being stripped away
by the GlobalDCE added by createStandardLTOPasses.
> 
> Sure, and why not?  Unused prototypes are not used for anything, they
won't
> turn up in the generated code for example.
They are not used this time. But why can't I generate additional code in the
same module after having run LTO?

Hi Christophe,
>>> Apparently, this depends on the oddly-named UnitAtATime parameter.
Set it to false. Try again. Suggestion: what about a comment explaining what
"UnitAtATime" means :-)
>>
>> If UnitAtATime is false this means that you are optimizing functions as
you are
>> generating them, rather than first generating all functions and other
globals
>> and only then optimizing.  It's basically a historical anachronism
coming from
>> the way GCC used to work.
>
> Thanks for explaining.
>
> Why not redefine it from historical anachronism to architectural flag, that
indicates you may generate more code with the same functions and globals later?
And then, you could pass the same flag to createStandardLTOPasses, and that
would skip the GlobalDCE passes.
LTO is for doing optimizations that are only valid when the module contains
everything that is needed to build the final executable.  So adding a flag to
say "not everything is there after all" makes no sense to me.
>>> Even then, my runtime function prorotypes are still being stripped
away by the GlobalDCE added by createStandardLTOPasses.
>>
>> Sure, and why not?  Unused prototypes are not used for anything, they
won't
>> turn up in the generated code for example.
>
> They are not used this time. But why can't I generate additional code
in the same module after having run LTO?
Because by definition LTO is for when everything is there!  Why don't you
create your own sequence of passes that does what you want rather than calling
createStandardLTOPasses?  One of the nice features of LLVM is that you can
run any passes you choose in any order you like.  That said, probably the reason
you are getting improved optimization is due to running the
"internalize" pass.
If so be warned that it is not safe to run internalize unless all functions and
their bodies are present in the module.  If you run internalize, optimize, and
then later add a few more functions to the module all kinds of nasty subtle
things can go wrong, not just those you saw with GlobalDCE.

Ciao, Duncan.

Hi Duncan,

On 7 janv. 2011, at 01:00, Duncan Sands wrote:
> LTO is for doing optimizations that are only valid when the module contains
> everything that is needed to build the final executable.  So adding a flag
to
> say "not everything is there after all" makes no sense to me.
And indeed, everything is there when I call LTO. The flag is not "not
everything is there after all", but "don't damage your
inputs".
> 
>>>> Even then, my runtime function prorotypes are still being
stripped away by the GlobalDCE added by createStandardLTOPasses.
>>> 
>>> Sure, and why not?  Unused prototypes are not used for anything,
they won't
>>> turn up in the generated code for example.
>> 
>> They are not used this time. But why can't I generate additional
code in the same module after having run LTO?
> 
> Because by definition LTO is for when everything is there!
And again, everything is there. I just want to be able to reuse the same inputs
later for incremental recompilation.
>  Why don't you create your own sequence of passes that does what you
want rather than calling createStandardLTOPasses?
As I wrote in an earlier reply, been there, done that. That forces me to
maintain a quasi-clone of standard passes.
>  One of the nice features of LLVM is that you can run any passes you choose
in any order you like.
Having standard LTO passes get in the way of incremental recompilation is not a
feature.
>  That said, probably the reason you are getting improved optimization is
due to running the "internalize" pass. If so be warned that it is not
safe to run internalize unless all functions and their bodies are present in the
module.
All functions and bodies are present, so that's not the issue. The issue is
that some inputs of LTO are destroyed as a side effect of running it.
> If you run internalize, optimize, and then later add a few more functions
to the module all kinds of nasty subtle things can go wrong, not just those you
saw with GlobalDCE.
Do you mean I should assume for example that the Functions I generated are also
irreversibly damaged? If so, what passes do I need to avoid if I want to
preserve the ability to do incremental recompilation?


Thanks for your help
Christophe