llvm dev - Nov 2016 - GlobalValue::AvailableExternallyLinkage

Thanks Mehdi.

My pass clones the functions within a module in order to have the original
function and an exact copy of the same function but with a different name, i.e.
sum() and sum_parallel().
After my pass I will run ThreadSanitizer instrumentation pass only on the new
copy of the functions, i.e. only the “_parallel” functions will be instrumented
by tsan.

In some programs that I am compiling, the functions such as atoi and atof get
cloned but I want to avoid this, and I noticed that only those functions have
GlobalValue::AvailableExternallyLinkage, so I was wondering if checking the 
linkage is enough to avoid those library functions or there could be situations
of functions with that linkage but that have the body implemented in the same
module.

Does it make sense now?

Thanks.
Simone
> On Nov 19, 2016, at 13:32, Mehdi Amini <mehdi.amini at apple.com>
wrote:
> 
> 
>> On Nov 19, 2016, at 12:00 PM, Simone Atzeni via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>> 
>> Hi,
>> 
>> could anybody explain what GlobalValue::AvailableExternallyLinkage
means?
> 
> It means that the function will not be codegen in this module, it will be
available at link time from another object.
> The IR is available for the purpose of inlining mainly.
> 
>> 
>> I implemented an instrumentation pass that creates a clone of a
function.
>> For some programs I noticed that also function such as “atoi”, “atof”,
“__strspn_c2”, etc. they get cloned even if I am not implementing them in the
module.
>> I would like to avoid cloning those functions, so I noticed that they
have GlobalValue::AvailableExternallyLinkage linkage.
>> Checking if they have GlobalValue::AvailableExternallyLinkage would it
be enough to ignore them?
> 
> I don’t really understand what you’re doing or why, so I can’t answer this
question.
> 
> — 
> Mehdi
>

> On Nov 19, 2016, at 12:44 PM, Simone Atzeni <simone.at at gmail.com>
wrote:
> 
> Thanks Mehdi.
> 
> My pass clones the functions within a module in order to have the original
function and an exact copy of the same function but with a different name, i.e.
sum() and sum_parallel().
> After my pass I will run ThreadSanitizer instrumentation pass only on the
new copy of the functions, i.e. only the “_parallel” functions will be
instrumented by tsan.
> 
> In some programs that I am compiling, the functions such as atoi and atof
get cloned but I want to avoid this, and I noticed that only those functions
have GlobalValue::AvailableExternallyLinkage, so I was wondering if checking the
linkage is enough to avoid those library functions or there could be situations
of functions with that linkage but that have the body implemented in the same
module.
> 
> Does it make sense now?
That gives me more context yes, now the question I’d have is maybe “why” you
don’t want to instrument these function as well here?
Depending on the underlying criteria that motivate it, the answer can be
different :)

— 
Mehdi

> 
>> On Nov 19, 2016, at 13:32, Mehdi Amini <mehdi.amini at apple.com>
wrote:
>> 
>> 
>>> On Nov 19, 2016, at 12:00 PM, Simone Atzeni via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>> 
>>> Hi,
>>> 
>>> could anybody explain what GlobalValue::AvailableExternallyLinkage
means?
>> 
>> It means that the function will not be codegen in this module, it will
be available at link time from another object.
>> The IR is available for the purpose of inlining mainly.
>> 
>>> 
>>> I implemented an instrumentation pass that creates a clone of a
function.
>>> For some programs I noticed that also function such as “atoi”,
“atof”, “__strspn_c2”, etc. they get cloned even if I am not implementing them
in the module.
>>> I would like to avoid cloning those functions, so I noticed that
they have GlobalValue::AvailableExternallyLinkage linkage.
>>> Checking if they have GlobalValue::AvailableExternallyLinkage would
it be enough to ignore them?
>> 
>> I don’t really understand what you’re doing or why, so I can’t answer
this question.
>> 
>> — 
>> Mehdi
>> 
>

Because what is happening is that if function “atoi” gets cloned I don’t have a
definition of “atoi_parallel” therefore I get undefined references when linking.
I just want to clone and instrument functions implemented in modules of my
program.
> On Nov 19, 2016, at 13:54, Mehdi Amini <mehdi.amini at apple.com>
wrote:
> 
> 
>> On Nov 19, 2016, at 12:44 PM, Simone Atzeni <simone.at at
gmail.com> wrote:
>> 
>> Thanks Mehdi.
>> 
>> My pass clones the functions within a module in order to have the
original function and an exact copy of the same function but with a different
name, i.e. sum() and sum_parallel().
>> After my pass I will run ThreadSanitizer instrumentation pass only on
the new copy of the functions, i.e. only the “_parallel” functions will be
instrumented by tsan.
>> 
>> In some programs that I am compiling, the functions such as atoi and
atof get cloned but I want to avoid this, and I noticed that only those
functions have GlobalValue::AvailableExternallyLinkage, so I was wondering if
checking the  linkage is enough to avoid those library functions or there could
be situations of functions with that linkage but that have the body implemented
in the same module.
>> 
>> Does it make sense now?
> 
> That gives me more context yes, now the question I’d have is maybe “why”
you don’t want to instrument these function as well here?
> Depending on the underlying criteria that motivate it, the answer can be
different :)
> 
> — 
> Mehdi
> 
> 
>> 
>>> On Nov 19, 2016, at 13:32, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>>> 
>>> 
>>>> On Nov 19, 2016, at 12:00 PM, Simone Atzeni via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> could anybody explain what
GlobalValue::AvailableExternallyLinkage means?
>>> 
>>> It means that the function will not be codegen in this module, it
will be available at link time from another object.
>>> The IR is available for the purpose of inlining mainly.
>>> 
>>>> 
>>>> I implemented an instrumentation pass that creates a clone of a
function.
>>>> For some programs I noticed that also function such as “atoi”,
“atof”, “__strspn_c2”, etc. they get cloned even if I am not implementing them
in the module.
>>>> I would like to avoid cloning those functions, so I noticed
that they have GlobalValue::AvailableExternallyLinkage linkage.
>>>> Checking if they have GlobalValue::AvailableExternallyLinkage
would it be enough to ignore them?
>>> 
>>> I don’t really understand what you’re doing or why, so I can’t
answer this question.
>>> 
>>> — 
>>> Mehdi
>>> 
>> 
>

On Sat, Nov 19, 2016 at 12:44 PM, Simone Atzeni via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Thanks Mehdi.
>
> My pass clones the functions within a module in order to have the original
> function and an exact copy of the same function but with a different name,
> i.e. sum() and sum_parallel().
> After my pass I will run ThreadSanitizer instrumentation pass only on the
> new copy of the functions, i.e. only the “_parallel” functions will be
> instrumented by tsan.
>
> In some programs that I am compiling, the functions such as atoi and atof
> get cloned but I want to avoid this, and I noticed that only those
> functions have GlobalValue::AvailableExternallyLinkage, so I was
> wondering if checking the  linkage is enough to avoid those library
> functions or there could be situations of functions with that linkage but
> that have the body implemented in the same module.
>
Checking for this linkage should be enough. If a function has
available_externally linkage, then some other module *must* provide an
equivalent, strong definition of that function, so you shouldn't need (or
want) to clone such functions.

The root of your problem is that you clone "atoi" into
"atoi_parallel", but
you leave the linkage as available_externally, which means that you are
also promising that some other module out there provides the symbol
"atoi_parallel". Obviously, that isn't the case, so you get linker
errors.

You could also resolve your problem by giving the parallel clones
'internal' linkage.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161121/a9f9faf3/attachment.html>

> On Nov 21, 2016, at 10:14 AM, Reid Kleckner <rnk at google.com>
wrote:
> 
> On Sat, Nov 19, 2016 at 12:44 PM, Simone Atzeni via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Thanks Mehdi.
> 
> My pass clones the functions within a module in order to have the original
function and an exact copy of the same function but with a different name, i.e.
sum() and sum_parallel().
> After my pass I will run ThreadSanitizer instrumentation pass only on the
new copy of the functions, i.e. only the “_parallel” functions will be
instrumented by tsan.
> 
> In some programs that I am compiling, the functions such as atoi and atof
get cloned but I want to avoid this, and I noticed that only those functions
have GlobalValue::AvailableExternallyLinkage, so I was wondering if checking the
linkage is enough to avoid those library functions or there could be situations
of functions with that linkage but that have the body implemented in the same
module.
> 
> Checking for this linkage should be enough. If a function has
available_externally linkage, then some other module *must* provide an
equivalent, strong definition of that function, so you shouldn't need (or
want) to clone such functions.
I disagree: depending on your use case you may want to instrument all the user
code of an application. Any available_externally function may be inlined in the
current module, in a non-instrumented form. The fact that it will be
instrumented in the original module is not enough.

— 
Mehdi

 
> 
> The root of your problem is that you clone "atoi" into
"atoi_parallel", but you leave the linkage as available_externally,
which means that you are also promising that some other module out there
provides the symbol "atoi_parallel". Obviously, that isn't the
case, so you get linker errors.
> 
> You could also resolve your problem by giving the parallel clones
'internal' linkage.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161121/0b943b07/attachment-0001.html>