llvm dev - Nov 2016 - RFC: Add an "interposible" linkage type (and implement -fsemantic-interposition)

I think that all makes sense. You're just adding the missing non-ODR
conterpart of 'external' linkage. I could imagine having "external
/
external_odr" linkage for example.

That said, do you think we should take the opportunity to split out a bit
for interposability so that we can kill off the *_odr linkage variants?
Today's non-ODR weak functions would look more like this:
  define weak interposable void @foo() { ret void }

We could probably preserve bitcode compatibility by continuing to use the
old combined linkage encoding. This is useful because we want the old weak
to decode as weak+interposable and the old weak_odr to decode as weak.

Some more prior discussion: https://reviews.llvm.org/D19995#423481

On Tue, Nov 29, 2016 at 8:01 AM, Hal Finkel via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi everyone,
>
> Clang/LLVM's support for ELF interposition is in a confusing state, and
I
> propose making a few (hopefully simple) adjustments in order to bring our
> model into a self-consistent state.
>
> The problem: On ELF systems, global symbols can be interposed. This means,
> for example, that calls to global functions in some (shared) library
> defined in that same library might end up being redirected to an
> implementation in some other library (or in the main executable). The most
> common reason for this is the use of LD_PRELOAD, but there are plenty of
> other ways to trigger interposition as well. As a result, it is technically
> inconsistent to inline any global function or do inter-procedural analysis
> on them because the implementation might be replaced by code with
> completely different behavior at runtme (or link time). Clang has never
> supported this (i.e. we do treat these functions as being eligible for
> inlining and perform IPA on them). GCC, on the other hand, has
> traditionally respected the possibility of ELF interposition and refrained
> from doing these things (at least when compiling with -fPIC).
>
> I believe that Clang/LLVM's current behavior is the most-useful
behavior
> and we should keep the current behavior (at least as a default). I do
> understand, however, that there are valid use cases for ELF interposition
> and places where we should allow it (e.g. when compiling certain system
> libraries). GCC recently added a flag
-fsemantic-interposition/-fno-semantic-interposition,
> where using -fno-semantic-interposition provides Clang/LLVM's behavior
of
> assuming that ELF interposition will not be used.
>
> It has been suggested that, to be self consistent, LLVM should emit global
> symbols with protected ELF visibility in cases where we've assumed that
ELF
> interposition won't happen. ELF protected visibility does seem to have
> exactly that meaning: A protected global symbol is externally visible but
> cannot be interposed. Unfortunately, as I understand it, on some major
> platforms (e.g. x86), protected-visibility symbols have a major flaw:
> Non-uniqueness of function pointers (i.e. the function pointer obtained to
> a function outside of the defining library might be different from the
> pointer obtained within the defining library). As a result, making this
> change might be practically prohibited (even if it makes sense in theory).
>
> Proposal:
>
>  1. Add a new linkage type, interposible, which is like external except
> that isInterposableLinkage will return true (thus preventing inlining, IPA,
> etc.). This is similar to weak linkage, in a sense, except that such
> symbols are never discarded and are not marked as weak for linking, etc.
>
>  2. Add -fsemantic-interposition/-fno-semantic-interposition to Clang.
> Default to -fno-semantic-interposition, but when -fsemantic-interposition
> is used, use interposible linkage for all functions where external linkage
> might otherwise have been used.
>
> Thoughts?
>
> Some useful links:
> http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html (the section
> on the -fno-semantic-interposition flag)
> https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html
>
> On some issues with ELF protected-visibility symbols:
> http://www.macieira.org/blog/2012/01/sorry-state-of-
> dynamic-libraries-on-linux/
> http://www.airs.com/blog/archives/307
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520
>
> Thanks again,
> Hal
>
> P.S. For some previous discussion on this, see below...
>
> ------------------------------
>
> *From: *"Hal Finkel via llvm-dev" <llvm-dev at
lists.llvm.org>
> *To: *"James Y Knight" <jyknight at google.com>
> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>
> *Sent: *Monday, February 29, 2016 9:50:15 AM
> *Subject: *Re: [llvm-dev] Possible soundness issue with
> available_externally (split from "RFC: Add guard intrinsics")
>
>
> ------------------------------
>
> *From: *"James Y Knight" <jyknight at google.com>
> *To: *"Hal Finkel" <hfinkel at anl.gov>
> *Cc: *"Sanjoy Das" <sanjoy at playingwithpointers.com>,
"llvm-dev" <
> llvm-dev at lists.llvm.org>
> *Sent: *Monday, February 29, 2016 9:31:24 AM
> *Subject: *Re: [llvm-dev] Possible soundness issue with
> available_externally (split from "RFC: Add guard intrinsics")
>
> On Feb 26, 2016 8:50 PM, "Hal Finkel" <hfinkel at anl.gov>
wrote:
>
>> *From: *"James Y Knight via llvm-dev" <llvm-dev at
lists.llvm.org>
>> *To: *"Sanjoy Das" <sanjoy at playingwithpointers.com>
>> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>
>> *Sent: *Thursday, February 25, 2016 1:41:43 PM
>> *Subject: *Re: [llvm-dev] Possible soundness issue with
>> available_externally (split from "RFC: Add guard intrinsics")
>>
>> While we're talking about this, I'd just mention again that the
same
>> issue arises for *normal* functions too, when linked into a shared
library:
>>    int foo() { return 1; }
>>    int bar() { return foo(); }
>>
>> Now, compare:
>>   clang -fPIC -O1 -S -o - test.c
>>   gcc -fPIC -O1 -S -o - test.c
>>
>> GCC will refuse to inline foo into bar, or use any information about
foo
>> in compiling bar, because foo is exported in the dynamic symbol table,
and
>> thus replaceable via symbol interposition.
>>
>> Clang assumes that you won't do that, or that you don't care
what happens
>> if you do. It will happily inline. And, in absense of inlining (e.g. if
foo
>> is too long to inline), clang will deduce function attributes about foo
and
>> rely on those in bar -- despite that the call goes through the PLT and
>> could in fact be an entirely different unrelated implementation (or,
for
>> that matter, a differently-optimized version of the same
implementation).
>>
>> Is that *really* okay?
>>
>> I'm comfortable with saying that symbol interposition falls outside
of
>> the model we have for the targeted system (at least by default), and
thus,
>> this is okay. We also don't model the possibility of someone
hex-editing
>> the binary ;)
>>
>
> I'm not really okay with it; the current behavior feels unprincipled.
>
> We have a visibility attribute which can be used to control this: On ELF
> systems, "default" visibililty allows interposition (unlike on
Darwin) --
> that is, it explicitly ALLOWS for replacing the symbol's definition.
The
> policy of "You can't replace the definition of the symbol, but it
is
> globally visible" is exactly what the "protected" visibility
mode is for.
>
> If we want to say that you can't interpose by default on ELF targets,
that
> would be a choice. Then, we should make the default symbol visibility
> "protected" instead of "default". But, continuing to
generate calls through
> the PLT -- which is only needed because the symbols might be replaced --
> while simultaneously making optimizations that are broken if they actually
> ARE replaced, seems kinda bogus.
>
> This makes sense, and I think you understand my concern here: Most
> programmers don't understand these issues, nor do they ever expect to
use
> dynamic interposition. They do expect, however, that the compiler has good
> IPA and will use the information it is provided effectively. I'd be
happy
> to make the default visibility protected, allowing us to continue
> optimizing well, and provide a principled behavior otherwise. Given, as you
> point out, this is the default on Darwin, is there experience from Darwin
> porting, or any other factors, that would indicate this would be a
hardship?
>
> Thanks again,
> Hal
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
> --
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161129/d2665d4e/attachment.html>

On Tue, Nov 29, 2016 at 9:15 AM Reid Kleckner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I think that all makes sense. You're just adding the missing non-ODR
> conterpart of 'external' linkage. I could imagine having
"external /
> external_odr" linkage for example.
>
> That said, do you think we should take the opportunity to split out a bit
> for interposability so that we can kill off the *_odr linkage variants?
> Today's non-ODR weak functions would look more like this:
>   define weak interposable void @foo() { ret void }
>
> We could probably preserve bitcode compatibility by continuing to use the
> old combined linkage encoding. This is useful because we want the old weak
> to decode as weak+interposable and the old weak_odr to decode as weak.
>
> Some more prior discussion: https://reviews.llvm.org/D19995#423481
>
>
+1

-eric

> On Tue, Nov 29, 2016 at 8:01 AM, Hal Finkel via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi everyone,
>
> Clang/LLVM's support for ELF interposition is in a confusing state, and
I
> propose making a few (hopefully simple) adjustments in order to bring our
> model into a self-consistent state.
>
> The problem: On ELF systems, global symbols can be interposed. This means,
> for example, that calls to global functions in some (shared) library
> defined in that same library might end up being redirected to an
> implementation in some other library (or in the main executable). The most
> common reason for this is the use of LD_PRELOAD, but there are plenty of
> other ways to trigger interposition as well. As a result, it is technically
> inconsistent to inline any global function or do inter-procedural analysis
> on them because the implementation might be replaced by code with
> completely different behavior at runtme (or link time). Clang has never
> supported this (i.e. we do treat these functions as being eligible for
> inlining and perform IPA on them). GCC, on the other hand, has
> traditionally respected the possibility of ELF interposition and refrained
> from doing these things (at least when compiling with -fPIC).
>
> I believe that Clang/LLVM's current behavior is the most-useful
behavior
> and we should keep the current behavior (at least as a default). I do
> understand, however, that there are valid use cases for ELF interposition
> and places where we should allow it (e.g. when compiling certain system
> libraries). GCC recently added a flag
> -fsemantic-interposition/-fno-semantic-interposition, where using
> -fno-semantic-interposition provides Clang/LLVM's behavior of assuming
that
> ELF interposition will not be used.
>
> It has been suggested that, to be self consistent, LLVM should emit global
> symbols with protected ELF visibility in cases where we've assumed that
ELF
> interposition won't happen. ELF protected visibility does seem to have
> exactly that meaning: A protected global symbol is externally visible but
> cannot be interposed. Unfortunately, as I understand it, on some major
> platforms (e.g. x86), protected-visibility symbols have a major flaw:
> Non-uniqueness of function pointers (i.e. the function pointer obtained to
> a function outside of the defining library might be different from the
> pointer obtained within the defining library). As a result, making this
> change might be practically prohibited (even if it makes sense in theory).
>
> Proposal:
>
>  1. Add a new linkage type, interposible, which is like external except
> that isInterposableLinkage will return true (thus preventing inlining, IPA,
> etc.). This is similar to weak linkage, in a sense, except that such
> symbols are never discarded and are not marked as weak for linking, etc.
>
>  2. Add -fsemantic-interposition/-fno-semantic-interposition to Clang.
> Default to -fno-semantic-interposition, but when -fsemantic-interposition
> is used, use interposible linkage for all functions where external linkage
> might otherwise have been used.
>
> Thoughts?
>
> Some useful links:
> http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html (the section
> on the -fno-semantic-interposition flag)
> https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html
>
> On some issues with ELF protected-visibility symbols:
>
>
http://www.macieira.org/blog/2012/01/sorry-state-of-dynamic-libraries-on-linux/
> http://www.airs.com/blog/archives/307
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520
>
> Thanks again,
> Hal
>
> P.S. For some previous discussion on this, see below...
>
> ------------------------------
>
> *From: *"Hal Finkel via llvm-dev" <llvm-dev at
lists.llvm.org>
> *To: *"James Y Knight" <jyknight at google.com>
> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>
> *Sent: *Monday, February 29, 2016 9:50:15 AM
> *Subject: *Re: [llvm-dev] Possible soundness issue with
> available_externally (split from "RFC: Add guard intrinsics")
>
>
> ------------------------------
>
> *From: *"James Y Knight" <jyknight at google.com>
> *To: *"Hal Finkel" <hfinkel at anl.gov>
> *Cc: *"Sanjoy Das" <sanjoy at playingwithpointers.com>,
"llvm-dev" <
> llvm-dev at lists.llvm.org>
> *Sent: *Monday, February 29, 2016 9:31:24 AM
> *Subject: *Re: [llvm-dev] Possible soundness issue with
> available_externally (split from "RFC: Add guard intrinsics")
>
> On Feb 26, 2016 8:50 PM, "Hal Finkel" <hfinkel at anl.gov>
wrote:
>
> *From: *"James Y Knight via llvm-dev" <llvm-dev at
lists.llvm.org>
> *To: *"Sanjoy Das" <sanjoy at playingwithpointers.com>
> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>
> *Sent: *Thursday, February 25, 2016 1:41:43 PM
> *Subject: *Re: [llvm-dev] Possible soundness issue with
> available_externally (split from "RFC: Add guard intrinsics")
>
> While we're talking about this, I'd just mention again that the
same issue
> arises for *normal* functions too, when linked into a shared library:
>    int foo() { return 1; }
>    int bar() { return foo(); }
>
> Now, compare:
>   clang -fPIC -O1 -S -o - test.c
>   gcc -fPIC -O1 -S -o - test.c
>
> GCC will refuse to inline foo into bar, or use any information about foo
> in compiling bar, because foo is exported in the dynamic symbol table, and
> thus replaceable via symbol interposition.
>
> Clang assumes that you won't do that, or that you don't care what
happens
> if you do. It will happily inline. And, in absense of inlining (e.g. if foo
> is too long to inline), clang will deduce function attributes about foo and
> rely on those in bar -- despite that the call goes through the PLT and
> could in fact be an entirely different unrelated implementation (or, for
> that matter, a differently-optimized version of the same implementation).
>
> Is that *really* okay?
>
> I'm comfortable with saying that symbol interposition falls outside of
the
> model we have for the targeted system (at least by default), and thus, this
> is okay. We also don't model the possibility of someone hex-editing the
> binary ;)
>
>
> I'm not really okay with it; the current behavior feels unprincipled.
>
> We have a visibility attribute which can be used to control this: On ELF
> systems, "default" visibililty allows interposition (unlike on
Darwin) --
> that is, it explicitly ALLOWS for replacing the symbol's definition.
The
> policy of "You can't replace the definition of the symbol, but it
is
> globally visible" is exactly what the "protected" visibility
mode is for.
>
> If we want to say that you can't interpose by default on ELF targets,
that
> would be a choice. Then, we should make the default symbol visibility
> "protected" instead of "default". But, continuing to
generate calls through
> the PLT -- which is only needed because the symbols might be replaced --
> while simultaneously making optimizations that are broken if they actually
> ARE replaced, seems kinda bogus.
>
> This makes sense, and I think you understand my concern here: Most
> programmers don't understand these issues, nor do they ever expect to
use
> dynamic interposition. They do expect, however, that the compiler has good
> IPA and will use the information it is provided effectively. I'd be
happy
> to make the default visibility protected, allowing us to continue
> optimizing well, and provide a principled behavior otherwise. Given, as you
> point out, this is the default on Darwin, is there experience from Darwin
> porting, or any other factors, that would indicate this would be a
hardship?
>
> Thanks again,
> Hal
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
> --
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161129/f89e65da/attachment.html>

----- Original Message -----
> From: "Eric Christopher" <echristo at gmail.com>
> To: "Reid Kleckner" <rnk at google.com>, "Hal
Finkel" <hfinkel at anl.gov>
> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>
> Sent: Tuesday, November 29, 2016 11:34:56 AM
> Subject: Re: [llvm-dev] RFC: Add an "interposible" linkage type
(and
> implement -fsemantic-interposition)
> On Tue, Nov 29, 2016 at 9:15 AM Reid Kleckner via llvm-dev <
> llvm-dev at lists.llvm.org > wrote:
> > I think that all makes sense. You're just adding the missing
> > non-ODR
> > conterpart of 'external' linkage. I could imagine having
"external
> > /
> > external_odr" linkage for example.
> 
> > That said, do you think we should take the opportunity to split out
> > a
> > bit for interposability so that we can kill off the *_odr linkage
> > variants? Today's non-ODR weak functions would look more like
this:
> 
> > define weak interposable void @foo() { ret void }
> 
> > We could probably preserve bitcode compatibility by continuing to
> > use
> > the old combined linkage encoding. This is useful because we want
> > the old weak to decode as weak+interposable and the old weak_odr to
> > decode as weak.
> 
> > Some more prior discussion: https://reviews.llvm.org/D19995#423481
> 
> +1
So we're all on the same page, our current encodings are: 

External: 0 
WeakAny: 16 
Appending: 2 
Internal: 3 
LinkOnceAny: 18 
ExternalWeak: 7 
Common: 8 
Private: 9 
WeakODR: 17 
LinkOnceODR: 19 
AvailableExternally: 12 

And the new encodings would be: 

External: 0 
External + Interposible: 20 
Weak + Interposible: 16 
Appending: 2 [interposible N/A] 
Internal: 3 [never interposible] 
LinkOnce + Interposible: 18 
ExternalWeak: 7 [always interposible] 
Common: 8 [interposible N/A] 
Private: 9 [never interposible] 
Weak: 17 
LinkOnce: 19 
AvailableExternally: 12 [never interposible] 

With the mappings: 

[Old] -> [New] 
External -> External 
WeakAny -> Weak + Interposible 
LinkOnceAny -> LinkOnce + Interposible 
WeakODR -> Weak 
LinkOnceODR -> LinkOnce 

Is this what you had in mind? 

Thanks again, 
Hal 
> -eric
> > On Tue, Nov 29, 2016 at 8:01 AM, Hal Finkel via llvm-dev <
> > llvm-dev at lists.llvm.org > wrote:
> 
> > > Hi everyone,
> > 
> 
> > > Clang/LLVM's support for ELF interposition is in a confusing
> > > state,
> > > and I propose making a few (hopefully simple) adjustments in
> > > order
> > > to bring our model into a self-consistent state.
> > 
> 
> > > The problem: On ELF systems, global symbols can be interposed.
> > > This
> > > means, for example, that calls to global functions in some
> > > (shared)
> > > library defined in that same library might end up being
> > > redirected
> > > to an implementation in some other library (or in the main
> > > executable). The most common reason for this is the use of
> > > LD_PRELOAD, but there are plenty of other ways to trigger
> > > interposition as well. As a result, it is technically
> > > inconsistent
> > > to inline any global function or do inter-procedural analysis on
> > > them because the implementation might be replaced by code with
> > > completely different behavior at runtme (or link time). Clang has
> > > never supported this (i.e. we do treat these functions as being
> > > eligible for inlining and perform IPA on them). GCC, on the other
> > > hand, has traditionally respected the possibility of ELF
> > > interposition and refrained from doing these things (at least
> > > when
> > > compiling with -fPIC).
> > 
> 
> > > I believe that Clang/LLVM's current behavior is the
most-useful
> > > behavior and we should keep the current behavior (at least as a
> > > default). I do understand, however, that there are valid use
> > > cases
> > > for ELF interposition and places where we should allow it (e.g.
> > > when
> > > compiling certain system libraries). GCC recently added a flag
> > > -fsemantic-interposition/-fno-semantic-interposition, where using
> > > -fno-semantic-interposition provides Clang/LLVM's behavior of
> > > assuming that ELF interposition will not be used.
> > 
> 
> > > It has been suggested that, to be self consistent, LLVM should
> > > emit
> > > global symbols with protected ELF visibility in cases where
we've
> > > assumed that ELF interposition won't happen. ELF protected
> > > visibility does seem to have exactly that meaning: A protected
> > > global symbol is externally visible but cannot be interposed.
> > > Unfortunately, as I understand it, on some major platforms (e.g.
> > > x86), protected-visibility symbols have a major flaw:
> > > Non-uniqueness
> > > of function pointers (i.e. the function pointer obtained to a
> > > function outside of the defining library might be different from
> > > the
> > > pointer obtained within the defining library). As a result,
> > > making
> > > this change might be practically prohibited (even if it makes
> > > sense
> > > in theory).
> > 
> 
> > > Proposal:
> > 
> 
> > > 1. Add a new linkage type, interposible, which is like external
> > > except that isInterposableLinkage will return true (thus
> > > preventing
> > > inlining, IPA, etc.). This is similar to weak linkage, in a
> > > sense,
> > > except that such symbols are never discarded and are not marked
> > > as
> > > weak for linking, etc.
> > 
> 
> > > 2. Add -fsemantic-interposition/-fno-semantic-interposition to
> > > Clang.
> > > Default to -fno-semantic-interposition, but when
> > > -fsemantic-interposition is used, use interposible linkage for
> > > all
> > > functions where external linkage might otherwise have been used.
> > 
> 
> > > Thoughts?
> > 
> 
> > > Some useful links:
> > 
> 
> > > http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html (the
> > > section on the -fno-semantic-interposition flag)
> > 
> 
> > > https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html
> > 
> 
> > > On some issues with ELF protected-visibility symbols:
> > 
> 
> > >
http://www.macieira.org/blog/2012/01/sorry-state-of-dynamic-libraries-on-linux/
> > 
> 
> > > http://www.airs.com/blog/archives/307
> > 
> 
> > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520
> > 
> 
> > > Thanks again,
> > 
> 
> > > Hal
> > 
> 
> > > P.S. For some previous discussion on this, see below...
> > 
> 
> > > > From: "Hal Finkel via llvm-dev" < llvm-dev at
lists.llvm.org >
> > > 
> > 
> 
> > > > To: "James Y Knight" < jyknight at google.com
>
> > > 
> > 
> 
> > > > Cc: "llvm-dev" < llvm-dev at lists.llvm.org
>
> > > 
> > 
> 
> > > > Sent: Monday, February 29, 2016 9:50:15 AM
> > > 
> > 
> 
> > > > Subject: Re: [llvm-dev] Possible soundness issue with
> > > > available_externally (split from "RFC: Add guard
intrinsics")
> > > 
> > 
> 
> > > > > From: "James Y Knight" < jyknight at
google.com >
> > > > 
> > > 
> > 
> 
> > > > > To: "Hal Finkel" < hfinkel at anl.gov >
> > > > 
> > > 
> > 
> 
> > > > > Cc: "Sanjoy Das" < sanjoy at
playingwithpointers.com >,
> > > > > "llvm-dev"
> > > > > <
> > > > > llvm-dev at lists.llvm.org >
> > > > 
> > > 
> > 
> 
> > > > > Sent: Monday, February 29, 2016 9:31:24 AM
> > > > 
> > > 
> > 
> 
> > > > > Subject: Re: [llvm-dev] Possible soundness issue with
> > > > > available_externally (split from "RFC: Add guard
intrinsics")
> > > > 
> > > 
> > 
> 
> > > > > On Feb 26, 2016 8:50 PM, "Hal Finkel" <
hfinkel at anl.gov >
> > > > > wrote:
> > > > 
> > > 
> > 
> 
> > > > > > > From: "James Y Knight via llvm-dev"
<
> > > > > > > llvm-dev at lists.llvm.org
> > > > > > > >
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > To: "Sanjoy Das" < sanjoy at
playingwithpointers.com >
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Cc: "llvm-dev" < llvm-dev at
lists.llvm.org >
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Sent: Thursday, February 25, 2016 1:41:43 PM
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Subject: Re: [llvm-dev] Possible soundness
issue with
> > > > > > > available_externally (split from "RFC:
Add guard
> > > > > > > intrinsics")
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > While we're talking about this, I'd
just mention again
> > > > > > > that
> > > > > > > the
> > > > > > > same
> > > > > > > issue arises for *normal* functions too, when
linked into
> > > > > > > a
> > > > > > > shared
> > > > > > > library:
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > int foo() { return 1; }
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > int bar() { return foo(); }
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Now, compare:
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > clang -fPIC -O1 -S -o - test.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > gcc -fPIC -O1 -S -o - test.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > GCC will refuse to inline foo into bar, or
use any
> > > > > > > information
> > > > > > > about
> > > > > > > foo in compiling bar, because foo is exported
in the
> > > > > > > dynamic
> > > > > > > symbol
> > > > > > > table, and thus replaceable via symbol
interposition.
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Clang assumes that you won't do that, or
that you don't
> > > > > > > care
> > > > > > > what
> > > > > > > happens if you do. It will happily inline.
And, in
> > > > > > > absense
> > > > > > > of
> > > > > > > inlining (e.g. if foo is too long to inline),
clang will
> > > > > > > deduce
> > > > > > > function attributes about foo and rely on
those in bar --
> > > > > > > despite
> > > > > > > that the call goes through the PLT and could
in fact be
> > > > > > > an
> > > > > > > entirely
> > > > > > > different unrelated implementation (or, for
that matter,
> > > > > > > a
> > > > > > > differently-optimized version of the same
> > > > > > > implementation).
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Is that *really* okay?
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > I'm comfortable with saying that symbol
interposition falls
> > > > > > outside
> > > > > > of the model we have for the targeted system (at
least by
> > > > > > default),
> > > > > > and thus, this is okay. We also don't model
the possibility
> > > > > > of
> > > > > > someone hex-editing the binary ;)
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > I'm not really okay with it; the current behavior
feels
> > > > > unprincipled.
> > > > 
> > > 
> > 
> 
> > > > > We have a visibility attribute which can be used to
control
> > > > > this:
> > > > > On
> > > > > ELF systems, "default" visibililty allows
interposition
> > > > > (unlike
> > > > > on
> > > > > Darwin) -- that is, it explicitly ALLOWS for replacing
the
> > > > > symbol's
> > > > > definition. The policy of "You can't replace
the definition
> > > > > of
> > > > > the
> > > > > symbol, but it is globally visible" is exactly
what the
> > > > > "protected"
> > > > > visibility mode is for.
> > > > 
> > > 
> > 
> 
> > > > > If we want to say that you can't interpose by
default on ELF
> > > > > targets,
> > > > > that would be a choice. Then, we should make the
default
> > > > > symbol
> > > > > visibility "protected" instead of
"default". But, continuing
> > > > > to
> > > > > generate calls through the PLT -- which is only needed
> > > > > because
> > > > > the
> > > > > symbols might be replaced -- while simultaneously
making
> > > > > optimizations that are broken if they actually ARE
replaced,
> > > > > seems
> > > > > kinda bogus.
> > > > 
> > > 
> > 
> 
> > > > This makes sense, and I think you understand my concern
here:
> > > > Most
> > > > programmers don't understand these issues, nor do they
ever
> > > > expect
> > > > to use dynamic interposition. They do expect, however, that
the
> > > > compiler has good IPA and will use the information it is
> > > > provided
> > > > effectively. I'd be happy to make the default visibility
> > > > protected,
> > > > allowing us to continue optimizing well, and provide a
> > > > principled
> > > > behavior otherwise. Given, as you point out, this is the
> > > > default
> > > > on
> > > > Darwin, is there experience from Darwin porting, or any
other
> > > > factors, that would indicate this would be a hardship?
> > > 
> > 
> 
> > > > Thanks again,
> > > 
> > 
> 
> > > > Hal
> > > 
> > 
> 
> > > > --
> > > 
> > 
> 
> > > > Hal Finkel
> > > 
> > 
> 
> > > > Assistant Computational Scientist
> > > 
> > 
> 
> > > > Leadership Computing Facility
> > > 
> > 
> 
> > > > Argonne National Laboratory
> > > 
> > 
> 
> > > > _______________________________________________
> > > 
> > 
> 
> > > > LLVM Developers mailing list
> > > 
> > 
> 
> > > > llvm-dev at lists.llvm.org
> > > 
> > 
> 
> > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > > 
> > 
> 
> > > --
> > 
> 
> > > Hal Finkel
> > 
> 
> > > Lead, Compiler Technology and Programming Languages
> > 
> 
> > > Leadership Computing Facility
> > 
> 
> > > Argonne National Laboratory
> > 
> 
> > > _______________________________________________
> > 
> 
> > > LLVM Developers mailing list
> > 
> 
> > > llvm-dev at lists.llvm.org
> > 
> 
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > 
> 
> > _______________________________________________
> 
> > LLVM Developers mailing list
> 
> > llvm-dev at lists.llvm.org
> 
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
-- 

Hal Finkel 
Lead, Compiler Technology and Programming Languages 
Leadership Computing Facility 
Argonne National Laboratory 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161129/3029d82a/attachment-0001.html>

> On Nov 29, 2016, at 9:14 AM, Reid Kleckner via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> I think that all makes sense. You're just adding the missing non-ODR
conterpart of 'external' linkage. I could imagine having "external
/ external_odr" linkage for example.
> 
> That said, do you think we should take the opportunity to split out a bit
for interposability so that we can kill off the *_odr linkage variants?
Today's non-ODR weak functions would look more like this:
>   define weak interposable void @foo() { ret void }
> 
> We could probably preserve bitcode compatibility by continuing to use the
old combined linkage encoding. This is useful because we want the old weak to
decode as weak+interposable and the old weak_odr to decode as weak.
> 
> Some more prior discussion: https://reviews.llvm.org/D19995#423481
<https://reviews.llvm.org/D19995#423481>
See also the open PR: https://llvm.org/bugs/show_bug.cgi?id=23501

— 
Mehdi
> 
> On Tue, Nov 29, 2016 at 8:01 AM, Hal Finkel via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hi everyone,
> 
> Clang/LLVM's support for ELF interposition is in a confusing state, and
I propose making a few (hopefully simple) adjustments in order to bring our
model into a self-consistent state.
> 
> The problem: On ELF systems, global symbols can be interposed. This means,
for example, that calls to global functions in some (shared) library defined in
that same library might end up being redirected to an implementation in some
other library (or in the main executable). The most common reason for this is
the use of LD_PRELOAD, but there are plenty of other ways to trigger
interposition as well. As a result, it is technically inconsistent to inline any
global function or do inter-procedural analysis on them because the
implementation might be replaced by code with completely different behavior at
runtme (or link time). Clang has never supported this (i.e. we do treat these
functions as being eligible for inlining and perform IPA on them). GCC, on the
other hand, has traditionally respected the possibility of ELF interposition and
refrained from doing these things (at least when compiling with -fPIC).
> 
> I believe that Clang/LLVM's current behavior is the most-useful
behavior and we should keep the current behavior (at least as a default). I do
understand, however, that there are valid use cases for ELF interposition and
places where we should allow it (e.g. when compiling certain system libraries).
GCC recently added a flag -fsemantic-interposition/-fno-semantic-interposition,
where using -fno-semantic-interposition provides Clang/LLVM's behavior of
assuming that ELF interposition will not be used.
> 
> It has been suggested that, to be self consistent, LLVM should emit global
symbols with protected ELF visibility in cases where we've assumed that ELF
interposition won't happen. ELF protected visibility does seem to have
exactly that meaning: A protected global symbol is externally visible but cannot
be interposed. Unfortunately, as I understand it, on some major platforms (e.g.
x86), protected-visibility symbols have a major flaw: Non-uniqueness of function
pointers (i.e. the function pointer obtained to a function outside of the
defining library might be different from the pointer obtained within the
defining library). As a result, making this change might be practically
prohibited (even if it makes sense in theory).
> 
> Proposal:
> 
>  1. Add a new linkage type, interposible, which is like external except
that isInterposableLinkage will return true (thus preventing inlining, IPA,
etc.). This is similar to weak linkage, in a sense, except that such symbols are
never discarded and are not marked as weak for linking, etc.
> 
>  2. Add -fsemantic-interposition/-fno-semantic-interposition to Clang.
Default to -fno-semantic-interposition, but when -fsemantic-interposition is
used, use interposible linkage for all functions where external linkage might
otherwise have been used.
> 
> Thoughts?
> 
> Some useful links:
> http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html
<http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html> (the section
on the -fno-semantic-interposition flag)
> https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html
<https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html>
> 
> On some issues with ELF protected-visibility symbols:
>
http://www.macieira.org/blog/2012/01/sorry-state-of-dynamic-libraries-on-linux/
<http://www.macieira.org/blog/2012/01/sorry-state-of-dynamic-libraries-on-linux/>
> http://www.airs.com/blog/archives/307
<http://www.airs.com/blog/archives/307>
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520>
> 
> Thanks again,
> Hal
> 
> P.S. For some previous discussion on this, see below...
> 
> From: "Hal Finkel via llvm-dev" <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
> To: "James Y Knight" <jyknight at google.com
<mailto:jyknight at google.com>>
> Cc: "llvm-dev" <llvm-dev at lists.llvm.org <mailto:llvm-dev
at lists.llvm.org>>
> Sent: Monday, February 29, 2016 9:50:15 AM
> Subject: Re: [llvm-dev] Possible soundness issue with available_externally
(split from "RFC: Add guard intrinsics")
> 
> 
> From: "James Y Knight" <jyknight at google.com
<mailto:jyknight at google.com>>
> To: "Hal Finkel" <hfinkel at anl.gov <mailto:hfinkel at
anl.gov>>
> Cc: "Sanjoy Das" <sanjoy at playingwithpointers.com
<mailto:sanjoy at playingwithpointers.com>>, "llvm-dev"
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
> Sent: Monday, February 29, 2016 9:31:24 AM
> Subject: Re: [llvm-dev] Possible soundness issue with available_externally
(split from "RFC: Add guard intrinsics")
> 
> On Feb 26, 2016 8:50 PM, "Hal Finkel" <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>> wrote:
> 
> From: "James Y Knight via llvm-dev" <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
> To: "Sanjoy Das" <sanjoy at playingwithpointers.com
<mailto:sanjoy at playingwithpointers.com>>
> Cc: "llvm-dev" <llvm-dev at lists.llvm.org <mailto:llvm-dev
at lists.llvm.org>>
> Sent: Thursday, February 25, 2016 1:41:43 PM
> Subject: Re: [llvm-dev] Possible soundness issue with available_externally
(split from "RFC: Add guard intrinsics")
> 
> While we're talking about this, I'd just mention again that the
same issue arises for *normal* functions too, when linked into a shared library:
>    int foo() { return 1; }
>    int bar() { return foo(); }
> 
> Now, compare:
>   clang -fPIC -O1 -S -o - test.c
>   gcc -fPIC -O1 -S -o - test.c
> 
> GCC will refuse to inline foo into bar, or use any information about foo in
compiling bar, because foo is exported in the dynamic symbol table, and thus
replaceable via symbol interposition.
> 
> Clang assumes that you won't do that, or that you don't care what
happens if you do. It will happily inline. And, in absense of inlining (e.g. if
foo is too long to inline), clang will deduce function attributes about foo and
rely on those in bar -- despite that the call goes through the PLT and could in
fact be an entirely different unrelated implementation (or, for that matter, a
differently-optimized version of the same implementation).
> 
> Is that *really* okay?
> 
> I'm comfortable with saying that symbol interposition falls outside of
the model we have for the targeted system (at least by default), and thus, this
is okay. We also don't model the possibility of someone hex-editing the
binary ;)
> 
> I'm not really okay with it; the current behavior feels unprincipled.
> 
> We have a visibility attribute which can be used to control this: On ELF
systems, "default" visibililty allows interposition (unlike on Darwin)
-- that is, it explicitly ALLOWS for replacing the symbol's definition. The
policy of "You can't replace the definition of the symbol, but it is
globally visible" is exactly what the "protected" visibility mode
is for.
> 
> If we want to say that you can't interpose by default on ELF targets,
that would be a choice. Then, we should make the default symbol visibility
"protected" instead of "default". But, continuing to
generate calls through the PLT -- which is only needed because the symbols might
be replaced -- while simultaneously making optimizations that are broken if they
actually ARE replaced, seems kinda bogus.
> 
> This makes sense, and I think you understand my concern here: Most
programmers don't understand these issues, nor do they ever expect to use
dynamic interposition. They do expect, however, that the compiler has good IPA
and will use the information it is provided effectively. I'd be happy to
make the default visibility protected, allowing us to continue optimizing well,
and provide a principled behavior otherwise. Given, as you point out, this is
the default on Darwin, is there experience from Darwin porting, or any other
factors, that would indicate this would be a hardship?
> 
> Thanks again,
> Hal
> 
> -- 
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> 
> -- 
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161129/4c013852/attachment.html>

> On Nov 29, 2016, at 11:22 AM, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
>> 
>> On Nov 29, 2016, at 9:14 AM, Reid Kleckner via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> I think that all makes sense. You're just adding the missing
non-ODR conterpart of 'external' linkage. I could imagine having
"external / external_odr" linkage for example.
>> 
>> That said, do you think we should take the opportunity to split out a
bit for interposability so that we can kill off the *_odr linkage variants?
Today's non-ODR weak functions would look more like this:
>>   define weak interposable void @foo() { ret void }
>> 
>> We could probably preserve bitcode compatibility by continuing to use
the old combined linkage encoding. This is useful because we want the old weak
to decode as weak+interposable and the old weak_odr to decode as weak.
>> 
>> Some more prior discussion: https://reviews.llvm.org/D19995#423481
<https://reviews.llvm.org/D19995#423481>
> See also the open PR: https://llvm.org/bugs/show_bug.cgi?id=23501
<https://llvm.org/bugs/show_bug.cgi?id=23501>
And another prior discussion and patch by Rafael (CC):
https://reviews.llvm.org/D20217 <https://reviews.llvm.org/D20217>

— 
Mehdi


> 
> — 
> Mehdi
> 
>> 
>> On Tue, Nov 29, 2016 at 8:01 AM, Hal Finkel via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> Hi everyone,
>> 
>> Clang/LLVM's support for ELF interposition is in a confusing state,
and I propose making a few (hopefully simple) adjustments in order to bring our
model into a self-consistent state.
>> 
>> The problem: On ELF systems, global symbols can be interposed. This
means, for example, that calls to global functions in some (shared) library
defined in that same library might end up being redirected to an implementation
in some other library (or in the main executable). The most common reason for
this is the use of LD_PRELOAD, but there are plenty of other ways to trigger
interposition as well. As a result, it is technically inconsistent to inline any
global function or do inter-procedural analysis on them because the
implementation might be replaced by code with completely different behavior at
runtme (or link time). Clang has never supported this (i.e. we do treat these
functions as being eligible for inlining and perform IPA on them). GCC, on the
other hand, has traditionally respected the possibility of ELF interposition and
refrained from doing these things (at least when compiling with -fPIC).
>> 
>> I believe that Clang/LLVM's current behavior is the most-useful
behavior and we should keep the current behavior (at least as a default). I do
understand, however, that there are valid use cases for ELF interposition and
places where we should allow it (e.g. when compiling certain system libraries).
GCC recently added a flag -fsemantic-interposition/-fno-semantic-interposition,
where using -fno-semantic-interposition provides Clang/LLVM's behavior of
assuming that ELF interposition will not be used.
>> 
>> It has been suggested that, to be self consistent, LLVM should emit
global symbols with protected ELF visibility in cases where we've assumed
that ELF interposition won't happen. ELF protected visibility does seem to
have exactly that meaning: A protected global symbol is externally visible but
cannot be interposed. Unfortunately, as I understand it, on some major platforms
(e.g. x86), protected-visibility symbols have a major flaw: Non-uniqueness of
function pointers (i.e. the function pointer obtained to a function outside of
the defining library might be different from the pointer obtained within the
defining library). As a result, making this change might be practically
prohibited (even if it makes sense in theory).
>> 
>> Proposal:
>> 
>>  1. Add a new linkage type, interposible, which is like external except
that isInterposableLinkage will return true (thus preventing inlining, IPA,
etc.). This is similar to weak linkage, in a sense, except that such symbols are
never discarded and are not marked as weak for linking, etc.
>> 
>>  2. Add -fsemantic-interposition/-fno-semantic-interposition to Clang.
Default to -fno-semantic-interposition, but when -fsemantic-interposition is
used, use interposible linkage for all functions where external linkage might
otherwise have been used.
>> 
>> Thoughts?
>> 
>> Some useful links:
>> http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html
<http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html> (the section
on the -fno-semantic-interposition flag)
>> https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html
<https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html>
>> 
>> On some issues with ELF protected-visibility symbols:
>>
http://www.macieira.org/blog/2012/01/sorry-state-of-dynamic-libraries-on-linux/
<http://www.macieira.org/blog/2012/01/sorry-state-of-dynamic-libraries-on-linux/>
>> http://www.airs.com/blog/archives/307
<http://www.airs.com/blog/archives/307>
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520>
>> 
>> Thanks again,
>> Hal
>> 
>> P.S. For some previous discussion on this, see below...
>> 
>> From: "Hal Finkel via llvm-dev" <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
>> To: "James Y Knight" <jyknight at google.com
<mailto:jyknight at google.com>>
>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
>> Sent: Monday, February 29, 2016 9:50:15 AM
>> Subject: Re: [llvm-dev] Possible soundness issue with
available_externally (split from "RFC: Add guard intrinsics")
>> 
>> 
>> From: "James Y Knight" <jyknight at google.com
<mailto:jyknight at google.com>>
>> To: "Hal Finkel" <hfinkel at anl.gov <mailto:hfinkel at
anl.gov>>
>> Cc: "Sanjoy Das" <sanjoy at playingwithpointers.com
<mailto:sanjoy at playingwithpointers.com>>, "llvm-dev"
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
>> Sent: Monday, February 29, 2016 9:31:24 AM
>> Subject: Re: [llvm-dev] Possible soundness issue with
available_externally (split from "RFC: Add guard intrinsics")
>> 
>> On Feb 26, 2016 8:50 PM, "Hal Finkel" <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>> wrote:
>> 
>> From: "James Y Knight via llvm-dev" <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
>> To: "Sanjoy Das" <sanjoy at playingwithpointers.com
<mailto:sanjoy at playingwithpointers.com>>
>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
>> Sent: Thursday, February 25, 2016 1:41:43 PM
>> Subject: Re: [llvm-dev] Possible soundness issue with
available_externally (split from "RFC: Add guard intrinsics")
>> 
>> While we're talking about this, I'd just mention again that the
same issue arises for *normal* functions too, when linked into a shared library:
>>    int foo() { return 1; }
>>    int bar() { return foo(); }
>> 
>> Now, compare:
>>   clang -fPIC -O1 -S -o - test.c
>>   gcc -fPIC -O1 -S -o - test.c
>> 
>> GCC will refuse to inline foo into bar, or use any information about
foo in compiling bar, because foo is exported in the dynamic symbol table, and
thus replaceable via symbol interposition.
>> 
>> Clang assumes that you won't do that, or that you don't care
what happens if you do. It will happily inline. And, in absense of inlining
(e.g. if foo is too long to inline), clang will deduce function attributes about
foo and rely on those in bar -- despite that the call goes through the PLT and
could in fact be an entirely different unrelated implementation (or, for that
matter, a differently-optimized version of the same implementation).
>> 
>> Is that *really* okay?
>> 
>> I'm comfortable with saying that symbol interposition falls outside
of the model we have for the targeted system (at least by default), and thus,
this is okay. We also don't model the possibility of someone hex-editing the
binary ;)
>> 
>> I'm not really okay with it; the current behavior feels
unprincipled.
>> 
>> We have a visibility attribute which can be used to control this: On
ELF systems, "default" visibililty allows interposition (unlike on
Darwin) -- that is, it explicitly ALLOWS for replacing the symbol's
definition. The policy of "You can't replace the definition of the
symbol, but it is globally visible" is exactly what the
"protected" visibility mode is for.
>> 
>> If we want to say that you can't interpose by default on ELF
targets, that would be a choice. Then, we should make the default symbol
visibility "protected" instead of "default". But, continuing
to generate calls through the PLT -- which is only needed because the symbols
might be replaced -- while simultaneously making optimizations that are broken
if they actually ARE replaced, seems kinda bogus.
>> 
>> This makes sense, and I think you understand my concern here: Most
programmers don't understand these issues, nor do they ever expect to use
dynamic interposition. They do expect, however, that the compiler has good IPA
and will use the information it is provided effectively. I'd be happy to
make the default visibility protected, allowing us to continue optimizing well,
and provide a principled behavior otherwise. Given, as you point out, this is
the default on Darwin, is there experience from Darwin porting, or any other
factors, that would indicate this would be a hardship?
>> 
>> Thanks again,
>> Hal
>> 
>> -- 
>> Hal Finkel
>> Assistant Computational Scientist
>> Leadership Computing Facility
>> Argonne National Laboratory
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>> 
>> 
>> 
>> -- 
>> Hal Finkel
>> Lead, Compiler Technology and Programming Languages
>> Leadership Computing Facility
>> Argonne National Laboratory
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161129/2f44051f/attachment.html>