llvm dev - Dec 2015 - RFC: New function attribute HasInaccessibleState

>> In the case of user-defined allocation functions, the definitions for
those functions are available
>Are they? probably not unless you're in an LTO build.
Yes, I'm assuming an LTO build.
>Printf() is a very nasty one because it can actually affect a lot of
state. The %n modifier can cause an argument to be written to.
hence it would have HasInaccessibleState and ArgMemOnly set, but not
ReadNone or ReadOnly set.
>Yes. Definitions being available should only *increase* the set of
attributes that can be added to them, never decrease.
I agree with that. But what I meant was, during a compilation invocation,
we either have the definitions available or not. That means, the attributes
we set once (or not set if definition isn't available) does not change.
>That's in practice impossible to guarantee, both by the compiler and by
the programmer.
I'm not sure I understand this. Why would it be impossible for the compiler
to propagate this flag along the call graph upwards? As an example, malloc
has the flag set, and this is propagated to whoever calls malloc, and then
to whoever calls that function and so on.



  - Vaivaswatha

On Fri, Dec 4, 2015 at 10:24 PM, Krzysztof Parzyszek via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 12/4/2015 10:48 AM, Vaivaswatha Nagaraj via llvm-dev wrote:
>
>> This point however reminds me to add, functions that transitively call
>> functions with HasInaccessibleState must also have the flag set.
>>
>
> That's in practice impossible to guarantee, both by the compiler and by
> the programmer.
>
> -Krzysztof
>
> --
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted
> by The Linux Foundation
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151204/f045ff1c/attachment.html>

On 12/4/2015 11:21 AM, Vaivaswatha Nagaraj wrote:
>
>  >That's in practice impossible to guarantee, both by the compiler
and
> by the programmer.
> I'm not sure I understand this. Why would it be impossible for the
> compiler to propagate this flag along the call graph upwards? As an
> example, malloc has the flag set, and this is propagated to whoever
> calls malloc, and then to whoever calls that function and so on.
Most of the time you don't have the entire call graph information. 
Imagine that you are developing a module that is a part of a larger 
project.  Functions in the other modules would need to have their flags 
updated based on what your functions do.

If you want to have an attribute stating that a function behaves better 
than what the compiler would normally assume, then the callers may but 
don't have to have that attribute set.  If you require that attribute 
for correctness, you will run into problems.  The sentence "For example, 
printf would have the flag set, preventing two calls to printfs (without 
the return value being used) from being interchanged." and the rest of 
that paragraph suggests the latter.

-Krzysztof

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, 
hosted by The Linux Foundation

----- Original Message ----- 
> From: "Vaivaswatha Nagaraj via llvm-dev" <llvm-dev at
lists.llvm.org>
> To: "Krzysztof Parzyszek" <kparzysz at codeaurora.org>
> Cc: "LLVM Dev" <llvm-dev at lists.llvm.org>
> Sent: Friday, December 4, 2015 11:21:03 AM
> Subject: Re: [llvm-dev] RFC: New function attribute
> HasInaccessibleState
> >> In the case of user-defined allocation functions, the definitions
> >> for those functions are available
> >Are they? probably not unless you're in an LTO build.
> Yes, I'm assuming an LTO build.
The concerns around LTO here, while legitimate, apply only to a very-specific
kind of LTO: An LTO which includes the definitions of the libc. This is actually
quite tricky to support, semantically, and already breaks our malloc aliasing
assumptions. There are many legitimate uses of LLVM, both for
statically-compiled code and for JIT'd code, that depend on a visibility
boundary between certain core runtime services and the user code being compiled
to provide for effective optimization.

So, yes, this will break LTO when you include libc itself in the optimization
process. We already don't support this (we'd need, at least, to adjust
our malloc noalias assumptions, if not many other things). I don't think
this is a major concern.

I think we need to go back and look at the underlying use case (as I understand
it): GlobalAA should be able to figure out that calls to malloc/free don't
touch global variables visible to the optimizer. How do we address this problem?

Thanks again,
Hal

...
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

>Most of the time you don't have the entire call graph information.
Imagine
that you are developing a module that is a part of a larger project.
I now understand the concern. It looks to me that we will need to set the
flag by default to all functions whose definitions aren't available
(external), and then propagate from there on. I don't see any optimizations
being inhibited by such a setting, so it should be okay.
>I think we need to go back and look at the underlying use case (as I
understand it): GlobalAA should be able to figure out that calls to
malloc/free don't touch global variables visible to the optimizer. How do
we address this problem?
Yes, this is the primary concern. Most libc functions (including printf,
malloc, free) fall into the same category.


  - Vaivaswatha

On Fri, Dec 4, 2015 at 11:12 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> ----- Original Message -----
> > From: "Vaivaswatha Nagaraj via llvm-dev" <llvm-dev at
lists.llvm.org>
> > To: "Krzysztof Parzyszek" <kparzysz at codeaurora.org>
> > Cc: "LLVM Dev" <llvm-dev at lists.llvm.org>
> > Sent: Friday, December 4, 2015 11:21:03 AM
> > Subject: Re: [llvm-dev] RFC: New function attribute
> > HasInaccessibleState
>
> > >> In the case of user-defined allocation functions, the
definitions
> > >> for those functions are available
>
> > >Are they? probably not unless you're in an LTO build.
>
> > Yes, I'm assuming an LTO build.
>
> The concerns around LTO here, while legitimate, apply only to a
> very-specific kind of LTO: An LTO which includes the definitions of the
> libc. This is actually quite tricky to support, semantically, and already
> breaks our malloc aliasing assumptions. There are many legitimate uses of
> LLVM, both for statically-compiled code and for JIT'd code, that depend
on
> a visibility boundary between certain core runtime services and the user
> code being compiled to provide for effective optimization.
>
> So, yes, this will break LTO when you include libc itself in the
> optimization process. We already don't support this (we'd need, at
least,
> to adjust our malloc noalias assumptions, if not many other things). I
> don't think this is a major concern.
>
> I think we need to go back and look at the underlying use case (as I
> understand it): GlobalAA should be able to figure out that calls to
> malloc/free don't touch global variables visible to the optimizer. How
do
> we address this problem?
>
> Thanks again,
> Hal
>
> ...
>
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> --
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151204/d0ae8ad5/attachment.html>

On Dec 4, 2015, at 12:42 PM, Hal Finkel via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>>> In the case of user-defined allocation functions, the
definitions
>>>> for those functions are available
> 
>>> Are they? probably not unless you're in an LTO build.
> 
>> Yes, I'm assuming an LTO build.
> 
> The concerns around LTO here, while legitimate, apply only to a
very-specific kind of LTO: An LTO which includes the definitions of the libc.
This is actually quite tricky to support, semantically, and already breaks our
malloc aliasing assumptions. There are many legitimate uses of LLVM, both for
statically-compiled code and for JIT'd code, that depend on a visibility
boundary between certain core runtime services and the user code being compiled
to provide for effective optimization.
Don't a lot of people use not-shipped-with-libc malloc/free replacements?
While LTO'ing your program with libc itself would seem unlikely, I don't
think LTO'ing a program along with a malloc replacement is that unusual.
(Either with a fully custom malloc or with something like tcmalloc).