llvm dev - Sep 2020 - EmitTargetCodeForMemSet & LTO issue

Hi Teresa,

Thank you for the help, adding the "used" attribute worked just fine.
It made me realize that the memset function is never inlined by the LTO
optimization even without my implementation of the
"EmitTargetCodeForMemSet" method.
I supposed that the passes dealing with the memset function happen too
late, is that correct?

Thank you again,
Romaric

On Fri, Aug 28, 2020 at 5:51 PM Teresa Johnson <tejohnson at google.com>
wrote:
> ThinLTO's whole program analysis is driven off of the linker symbol
> resolution (at least in the case of lld, gold, bfd). Presumably when it
> gets into the ThinLTO link, the linker does not see any uses of your
> alternate memset and it is therefore removed as dead code. Have you tried
> marking it with __attribute__((used))? Another thing that might work is to
> mark it weak.
>
> Teresa
>
> On Fri, Aug 28, 2020 at 8:26 AM Romaric Jodin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi everyone,
>>
>> I have 2 implementations of "memset". A standard one, and
another one
>> optimized when the pointer and the size respect some specific
constraints.
>> I am able to choose the proper one in the
"EmitTargetCodeForMemSet"
>> method that I implemented for my backend.
>>
>> My issue is when I am compiling with the LTO optimisation, the linker
>> tells me that the optimized memset symbol is undefined ("*ld.lld:
error:
>> undefined symbol: __memset_opt*"). I've looked into theLTO
archive used
>> and I found both memset functions.
>>
>> It feels like when the compiler gets in
"EmitTargetCodeForMemSet", it has
>> already forgotten about the optimized memset function that was in the
>> archive because it was not used initially.
>>
>> Any idea why?
>>
>> Thanks in advance,
>> JODIN Romaric
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>

-- 
*Romaric JODIN*
UPMEM
*Software Engineer*
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200831/65acc4b7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logo signature mail 50x50.png
Type: image/png
Size: 1939 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200831/65acc4b7/attachment.png>

Are you compiling with -fno-builtin-memset? If not the memset calls are
replaced with an llvm builtin memset intrinsic call, which is expanded
after inlining. With -fno-builtin-memset the memset calls are left as
regular calls, since it tells clang that you are not using the builtin
version. However, if the definition is in another translation unit, and the
linker doesn't think that version is prevailing, it won't get imported
or
inlined by LTO/ThinLTO in any case.

Teresa

On Sun, Aug 30, 2020 at 11:11 PM Romaric Jodin <rjodin at upmem.com>
wrote:
> Hi Teresa,
>
> Thank you for the help, adding the "used" attribute worked just
fine.
> It made me realize that the memset function is never inlined by the LTO
> optimization even without my implementation of the
> "EmitTargetCodeForMemSet" method.
> I supposed that the passes dealing with the memset function happen too
> late, is that correct?
>
> Thank you again,
> Romaric
>
> On Fri, Aug 28, 2020 at 5:51 PM Teresa Johnson <tejohnson at
google.com>
> wrote:
>
>> ThinLTO's whole program analysis is driven off of the linker symbol
>> resolution (at least in the case of lld, gold, bfd). Presumably when it
>> gets into the ThinLTO link, the linker does not see any uses of your
>> alternate memset and it is therefore removed as dead code. Have you
tried
>> marking it with __attribute__((used))? Another thing that might work is
to
>> mark it weak.
>>
>> Teresa
>>
>> On Fri, Aug 28, 2020 at 8:26 AM Romaric Jodin via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi everyone,
>>>
>>> I have 2 implementations of "memset". A standard one, and
another one
>>> optimized when the pointer and the size respect some specific
constraints.
>>> I am able to choose the proper one in the
"EmitTargetCodeForMemSet"
>>> method that I implemented for my backend.
>>>
>>> My issue is when I am compiling with the LTO optimisation, the
linker
>>> tells me that the optimized memset symbol is undefined
("*ld.lld:
>>> error: undefined symbol: __memset_opt*"). I've looked into
theLTO
>>> archive used and I found both memset functions.
>>>
>>> It feels like when the compiler gets in
"EmitTargetCodeForMemSet", it
>>> has already forgotten about the optimized memset function that was
in the
>>> archive because it was not used initially.
>>>
>>> Any idea why?
>>>
>>> Thanks in advance,
>>> JODIN Romaric
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>>
>> --
>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>
>
>
> --
> *Romaric JODIN*
> UPMEM
> *Software Engineer*
>
>
-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200901/7344c77c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logo signature mail 50x50.png
Type: image/png
Size: 1939 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200901/7344c77c/attachment.png>

On 2020-09-01, Teresa Johnson via llvm-dev wrote:
>Are you compiling with -fno-builtin-memset? If not the memset calls are
>replaced with an llvm builtin memset intrinsic call, which is expanded
>after inlining. With -fno-builtin-memset the memset calls are left as
>regular calls, since it tells clang that you are not using the builtin
>version. However, if the definition is in another translation unit, and the
>linker doesn't think that version is prevailing, it won't get
imported or
>inlined by LTO/ThinLTO in any case.
>
>Teresa
Another possibility, if you arrange for EmitTargetCodeForMemSet to call
a customized library function which is not a registered libcall, it may
not be fetched if it is in an archive. See lld/ELF/Driver.cpp handleLibCall.
>On Sun, Aug 30, 2020 at 11:11 PM Romaric Jodin <rjodin at upmem.com>
wrote:
>
>> Hi Teresa,
>>
>> Thank you for the help, adding the "used" attribute worked
just fine.
>> It made me realize that the memset function is never inlined by the LTO
>> optimization even without my implementation of the
>> "EmitTargetCodeForMemSet" method.
>> I supposed that the passes dealing with the memset function happen too
>> late, is that correct?
>>
>> Thank you again,
>> Romaric
>>
>> On Fri, Aug 28, 2020 at 5:51 PM Teresa Johnson <tejohnson at
google.com>
>> wrote:
>>
>>> ThinLTO's whole program analysis is driven off of the linker
symbol
>>> resolution (at least in the case of lld, gold, bfd). Presumably
when it
>>> gets into the ThinLTO link, the linker does not see any uses of
your
>>> alternate memset and it is therefore removed as dead code. Have you
tried
>>> marking it with __attribute__((used))? Another thing that might
work is to
>>> mark it weak.
>>>
>>> Teresa
>>>
>>> On Fri, Aug 28, 2020 at 8:26 AM Romaric Jodin via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I have 2 implementations of "memset". A standard one,
and another one
>>>> optimized when the pointer and the size respect some specific
constraints.
>>>> I am able to choose the proper one in the
"EmitTargetCodeForMemSet"
>>>> method that I implemented for my backend.
>>>>
>>>> My issue is when I am compiling with the LTO optimisation, the
linker
>>>> tells me that the optimized memset symbol is undefined
("*ld.lld:
>>>> error: undefined symbol: __memset_opt*"). I've looked
into theLTO
>>>> archive used and I found both memset functions.
>>>>
>>>> It feels like when the compiler gets in
"EmitTargetCodeForMemSet", it
>>>> has already forgotten about the optimized memset function that
was in the
>>>> archive because it was not used initially.
>>>>
>>>> Any idea why?
>>>>
>>>> Thanks in advance,
>>>> JODIN Romaric
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
>>>
>>> --
>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>>
>>
>>
>> --
>> *Romaric JODIN*
>> UPMEM
>> *Software Engineer*
>>
>>
>
>-- 
>Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>File Name                       : logo_signature_mail_50x50.png
>File Size                       : 1939 bytes
>Image Size                      : 50x50
>_______________________________________________
>LLVM Developers mailing list
>llvm-dev at lists.llvm.org
>https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev