llvm dev - May 2018 - lld + ThinLTO + fprofile-generate causes duplicate symbol errors

LLD revision is r331862.  To add, I had initially tried it on r328903,
which also reproduced the issue.

On Wed, May 9, 2018 at 9:26 AM Pirama Arumuga Nainar <pirama at
google.com>
wrote:
> Hi Teresa,
>
> Thanks for looking into this.  I hadn't initially tried ToT, but it
> reproduces in ToT as well when I tried.
>
> $ ./clang --version
>
> clang version 7.0.0 (trunk 331879) (llvm/trunk 331888)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
> InstalledDir: /ssd2/pirama/llvm-upstream/llvm-release/bin/.
>
> $ ./ld.lld --version
> LLD 7.0.0 (https://git.llvm.org/git/lld.git
dc4977e104c7f33758ac6119d1528aafcd8a94dc) (compatible with GNU linkers)
>
>
> On Wed, May 9, 2018 at 6:44 AM Teresa Johnson <tejohnson at
google.com>
> wrote:
>
>>
>>
>> On Wed, May 9, 2018 at 6:43 AM Teresa Johnson <tejohnson at
google.com>
>> wrote:
>>
>>> Hi Pirama,
>>>
>>> I can't reproduce with either lld or gold, using a compiler
built from
>>> head. What version is your clang?
>>>
>> (and your lld)
>>
>>
>>>
>>> Thanks,
>>> Teresa
>>>
>>> On Tue, May 8, 2018 at 7:50 PM Pirama Arumuga Nainar via llvm-dev
<
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> The duplicate symbol errors are for __llvm_profile_filename and
__llvm_profile_raw_version.
>>>> IIUC, these are supposed to be weak symbols but Thin LTO seems
to break
>>>> this in some way.  This does't happen with gold or no LTO
or full LTO.
>>>>
>>>> $ cat > a.c
>>>> extern int foo();
>>>>
>>>> int main() {
>>>>   return foo();
>>>> }
>>>>
>>>> $ cat > b.c
>>>> int foo() {
>>>>   return 0;
>>>> }
>>>>
>>>> $ clang a.c -fprofile-generate -flto=thin -c
>>>> $ clang b.c -fprofile-generate -flto=thin -c
>>>> $ clang a.o b.o -fprofile-generate -flto=thin -fuse-ld=lld
>>>> ld.lld: error: duplicate symbol: __llvm_profile_filename
>>>> >>> defined at a.c
>>>> >>>            lto.tmp:(__llvm_profile_filename)
>>>> >>> defined at b.c
>>>> >>>           
lto.tmp:(.rodata.__llvm_profile_filename+0x0)
>>>>
>>>> ld.lld: error: duplicate symbol: __llvm_profile_raw_version
>>>> >>> defined at a.c
>>>> >>>            lto.tmp:(__llvm_profile_raw_version)
>>>> >>> defined at b.c
>>>> >>>           
lto.tmp:(.rodata.__llvm_profile_raw_version+0x0)
>>>> clang-7: error: linker command failed with exit code 1 (use -v
to see
>>>> invocation)
>>>> Makefile:10: recipe for target 'a.out' failed
>>>> make: *** [a.out] Error 1
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
>>>
>>> --
>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>>  408-460-2413
>>>
>>
>>
>> --
>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>  408-460-2413
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180509/75e9a8b3/attachment.html>

Sorry, operator error. I can reproduce now. Interestingly, this does not
reproduce using gold, and they utilize the same underlying LTO API. Let me
dig a little using save-temps and see where they diverge.
Teresa

On Wed, May 9, 2018 at 9:28 AM Pirama Arumuga Nainar <pirama at
google.com>
wrote:
> LLD revision is r331862.  To add, I had initially tried it on r328903,
> which also reproduced the issue.
>
> On Wed, May 9, 2018 at 9:26 AM Pirama Arumuga Nainar <pirama at
google.com>
> wrote:
>
>> Hi Teresa,
>>
>> Thanks for looking into this.  I hadn't initially tried ToT, but it
>> reproduces in ToT as well when I tried.
>>
>> $ ./clang --version
>>
>> clang version 7.0.0 (trunk 331879) (llvm/trunk 331888)
>> Target: x86_64-unknown-linux-gnu
>> Thread model: posix
>> InstalledDir: /ssd2/pirama/llvm-upstream/llvm-release/bin/.
>>
>> $ ./ld.lld --version
>> LLD 7.0.0 (https://git.llvm.org/git/lld.git
dc4977e104c7f33758ac6119d1528aafcd8a94dc) (compatible with GNU linkers)
>>
>>
>> On Wed, May 9, 2018 at 6:44 AM Teresa Johnson <tejohnson at
google.com>
>> wrote:
>>
>>>
>>>
>>> On Wed, May 9, 2018 at 6:43 AM Teresa Johnson <tejohnson at
google.com>
>>> wrote:
>>>
>>>> Hi Pirama,
>>>>
>>>> I can't reproduce with either lld or gold, using a compiler
built from
>>>> head. What version is your clang?
>>>>
>>> (and your lld)
>>>
>>>
>>>>
>>>> Thanks,
>>>> Teresa
>>>>
>>>> On Tue, May 8, 2018 at 7:50 PM Pirama Arumuga Nainar via
llvm-dev <
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>> The duplicate symbol errors are for __llvm_profile_filename
and __llvm_profile_raw_version.
>>>>> IIUC, these are supposed to be weak symbols but Thin LTO
seems to break
>>>>> this in some way.  This does't happen with gold or no
LTO or full LTO.
>>>>>
>>>>> $ cat > a.c
>>>>> extern int foo();
>>>>>
>>>>> int main() {
>>>>>   return foo();
>>>>> }
>>>>>
>>>>> $ cat > b.c
>>>>> int foo() {
>>>>>   return 0;
>>>>> }
>>>>>
>>>>> $ clang a.c -fprofile-generate -flto=thin -c
>>>>> $ clang b.c -fprofile-generate -flto=thin -c
>>>>> $ clang a.o b.o -fprofile-generate -flto=thin -fuse-ld=lld
>>>>> ld.lld: error: duplicate symbol: __llvm_profile_filename
>>>>> >>> defined at a.c
>>>>> >>>            lto.tmp:(__llvm_profile_filename)
>>>>> >>> defined at b.c
>>>>> >>>           
lto.tmp:(.rodata.__llvm_profile_filename+0x0)
>>>>>
>>>>> ld.lld: error: duplicate symbol: __llvm_profile_raw_version
>>>>> >>> defined at a.c
>>>>> >>>           
lto.tmp:(__llvm_profile_raw_version)
>>>>> >>> defined at b.c
>>>>> >>>           
lto.tmp:(.rodata.__llvm_profile_raw_version+0x0)
>>>>> clang-7: error: linker command failed with exit code 1 (use
-v to see
>>>>> invocation)
>>>>> Makefile:10: recipe for target 'a.out' failed
>>>>> make: *** [a.out] Error 1
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>
>>>>
>>>> --
>>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com
|
>>>>  408-460-2413
>>>>
>>>
>>>
>>> --
>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>>  408-460-2413
>>>
>>
-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |  408-460-2413
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180509/a93a9025/attachment.html>

Adding Peter to comment on the linker resolution issue.
>From adding save-temps, it looks like lld and gold are giving different
resolutions to the symbols, which is presumably creating this issue:

(first file is with lld and second is with gold)

$ diff a.out.resolution.txt gold/
4c4
< -r=a.o,__llvm_profile_raw_version,plx
---
> -r=a.o,__llvm_profile_raw_version,l
8,9c8,9
< -r=b.o,__llvm_profile_raw_version,x
< -r=b.o,__llvm_profile_filename,x
---
> -r=b.o,__llvm_profile_raw_version,l
> -r=b.o,__llvm_profile_filename,l
I.e. with lld they are all marked as exported.

Peter, any ideas?

Teresa


On Wed, May 9, 2018 at 9:52 AM Teresa Johnson <tejohnson at google.com>
wrote:
> Sorry, operator error. I can reproduce now. Interestingly, this does not
> reproduce using gold, and they utilize the same underlying LTO API. Let me
> dig a little using save-temps and see where they diverge.
> Teresa
>
> On Wed, May 9, 2018 at 9:28 AM Pirama Arumuga Nainar <pirama at
google.com>
> wrote:
>
>> LLD revision is r331862.  To add, I had initially tried it on r328903,
>> which also reproduced the issue.
>>
>> On Wed, May 9, 2018 at 9:26 AM Pirama Arumuga Nainar <pirama at
google.com>
>> wrote:
>>
>>> Hi Teresa,
>>>
>>> Thanks for looking into this.  I hadn't initially tried ToT,
but it
>>> reproduces in ToT as well when I tried.
>>>
>>> $ ./clang --version
>>>
>>> clang version 7.0.0 (trunk 331879) (llvm/trunk 331888)
>>> Target: x86_64-unknown-linux-gnu
>>> Thread model: posix
>>> InstalledDir: /ssd2/pirama/llvm-upstream/llvm-release/bin/.
>>>
>>> $ ./ld.lld --version
>>> LLD 7.0.0 (https://git.llvm.org/git/lld.git
dc4977e104c7f33758ac6119d1528aafcd8a94dc) (compatible with GNU linkers)
>>>
>>>
>>> On Wed, May 9, 2018 at 6:44 AM Teresa Johnson <tejohnson at
google.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Wed, May 9, 2018 at 6:43 AM Teresa Johnson <tejohnson at
google.com>
>>>> wrote:
>>>>
>>>>> Hi Pirama,
>>>>>
>>>>> I can't reproduce with either lld or gold, using a
compiler built from
>>>>> head. What version is your clang?
>>>>>
>>>> (and your lld)
>>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>> Teresa
>>>>>
>>>>> On Tue, May 8, 2018 at 7:50 PM Pirama Arumuga Nainar via
llvm-dev <
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> The duplicate symbol errors are for
__llvm_profile_filename and __llvm_profile_raw_version.
>>>>>> IIUC, these are supposed to be weak symbols but Thin
LTO seems to break
>>>>>> this in some way.  This does't happen with gold or
no LTO or full
>>>>>> LTO.
>>>>>>
>>>>>> $ cat > a.c
>>>>>> extern int foo();
>>>>>>
>>>>>> int main() {
>>>>>>   return foo();
>>>>>> }
>>>>>>
>>>>>> $ cat > b.c
>>>>>> int foo() {
>>>>>>   return 0;
>>>>>> }
>>>>>>
>>>>>> $ clang a.c -fprofile-generate -flto=thin -c
>>>>>> $ clang b.c -fprofile-generate -flto=thin -c
>>>>>> $ clang a.o b.o -fprofile-generate -flto=thin
-fuse-ld=lld
>>>>>> ld.lld: error: duplicate symbol:
__llvm_profile_filename
>>>>>> >>> defined at a.c
>>>>>> >>>           
lto.tmp:(__llvm_profile_filename)
>>>>>> >>> defined at b.c
>>>>>> >>>           
lto.tmp:(.rodata.__llvm_profile_filename+0x0)
>>>>>>
>>>>>> ld.lld: error: duplicate symbol:
__llvm_profile_raw_version
>>>>>> >>> defined at a.c
>>>>>> >>>           
lto.tmp:(__llvm_profile_raw_version)
>>>>>> >>> defined at b.c
>>>>>> >>>           
lto.tmp:(.rodata.__llvm_profile_raw_version+0x0)
>>>>>> clang-7: error: linker command failed with exit code 1
(use -v to see
>>>>>> invocation)
>>>>>> Makefile:10: recipe for target 'a.out' failed
>>>>>> make: *** [a.out] Error 1
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> llvm-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Teresa Johnson |  Software Engineer |  tejohnson at
google.com |
>>>>>  408-460-2413
>>>>>
>>>>
>>>>
>>>> --
>>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com
|
>>>>  408-460-2413
>>>>
>>>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>  408-460-2413
>

-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |  408-460-2413
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180509/91a82219/attachment.html>