llvm dev - Feb 2020 - LLD doesn't handle globals with appending linkage

Hello.

I'm posting this question here, because there seem to be no LLD-specific
mailing list. Sorry in advance if this is wrong one.

I compile two C source with following command:

clang -flto -o %name.bc %name.c

LLVM is augmented with my custom pass, which amongst other things create a
global with appending linkage:

@myvar = appending constant [1 x [1 x i8]*] ...

I also have another pass plugged into LTO pipeline, which turns this global
into internal one in the final module. I was hoping that LLD would first
link bitcodes like llvm-link, appending @myvar's from both modules into a
single one, then run my pass, and only then perform linking at object level.

However, LLD complains about duplicated symbol "myvar" and doesn't
even run
any passes.

I've tracked the problem down to BitcodeFile::parse() function from
https://github.com/llvm/llvm-project/blob/master/lld/COFF/InputFiles.cpp#L918
- it doesn't take linkage type into account.

Can anything be done about this problem? Or was my approach broken from the
beginning?

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200218/b74c152b/attachment.html>

I'm /guessing/ this might be related to the COFF support specifically
(perhaps COFF has no appending linkage support - in some cases LLVM IR
supports the union of all semantics so that different formats can be fully
expressed - but it means when targeting certain formats, some features are
inherently unusable because they don't map to anything on that platform)

If I were you I'd go see if there are any examples of appending linkage
being used when targeting the COFF platform (eg: go look at places where
Clang generates IR with appending linkage, see if any of them apply to
COFF, if they do see how they work through LTO - if there are no such
examples, look for non-COFF Examples of appending linkage and see what the
IR targeting COFF looks like in those examples, perhaps a different linkage
mechanism is used that you can use too)

(added Reid for familiarity with the COFF format & Ray for familiarity with
LLD)

On Tue, Feb 18, 2020 at 12:29 AM Gleb Popov via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello.
>
> I'm posting this question here, because there seem to be no
LLD-specific
> mailing list. Sorry in advance if this is wrong one.
>
> I compile two C source with following command:
>
> clang -flto -o %name.bc %name.c
>
> LLVM is augmented with my custom pass, which amongst other things create a
> global with appending linkage:
>
> @myvar = appending constant [1 x [1 x i8]*] ...
>
> I also have another pass plugged into LTO pipeline, which turns this
> global into internal one in the final module. I was hoping that LLD would
> first link bitcodes like llvm-link, appending @myvar's from both
modules
> into a single one, then run my pass, and only then perform linking at
> object level.
>
> However, LLD complains about duplicated symbol "myvar" and
doesn't even
> run any passes.
>
> I've tracked the problem down to BitcodeFile::parse() function from
>
https://github.com/llvm/llvm-project/blob/master/lld/COFF/InputFiles.cpp#L918
> - it doesn't take linkage type into account.
>
> Can anything be done about this problem? Or was my approach broken from
> the beginning?
>
> Thanks in advance.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200218/2ae74ed5/attachment.html>

On Tue, Feb 18, 2020 at 9:42 PM David Blaikie <dblaikie at gmail.com>
wrote:
> I'm /guessing/ this might be related to the COFF support specifically
> (perhaps COFF has no appending linkage support - in some cases LLVM IR
> supports the union of all semantics so that different formats can be fully
> expressed - but it means when targeting certain formats, some features are
> inherently unusable because they don't map to anything on that
platform)
>
Actually, I experience the same problem with ELF on FreeBSD.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200218/9f6ce40a/attachment.html>

On Tue, Feb 18, 2020 at 12:28 PM Gleb Popov <6yearold at gmail.com> wrote:
> Hello.
>
> I'm posting this question here, because there seem to be no
LLD-specific
> mailing list. Sorry in advance if this is wrong one.
>
> I compile two C source with following command:
>
> clang -flto -o %name.bc %name.c
>
> LLVM is augmented with my custom pass, which amongst other things create a
> global with appending linkage:
>
> @myvar = appending constant [1 x [1 x i8]*] ...
>
> I also have another pass plugged into LTO pipeline, which turns this
> global into internal one in the final module. I was hoping that LLD would
> first link bitcodes like llvm-link, appending @myvar's from both
modules
> into a single one, then run my pass, and only then perform linking at
> object level.
>
> However, LLD complains about duplicated symbol "myvar" and
doesn't even
> run any passes.
>
> I've tracked the problem down to BitcodeFile::parse() function from
>
https://github.com/llvm/llvm-project/blob/master/lld/COFF/InputFiles.cpp#L918
> - it doesn't take linkage type into account.
>
> Can anything be done about this problem? Or was my approach broken from
> the beginning?
>
> Thanks in advance.
>

The following patch has fixed the problem for me:

diff --git a/llvm/lib/Object/ModuleSymbolTable.cpp
b/llvm/lib/Object/ModuleSymbolTable.cpp
index 33ce7d8109f..d1d74fa54bf 100644
--- a/lib/Object/ModuleSymbolTable.cpp
+++ b/lib/Object/ModuleSymbolTable.cpp
@@ -201,7 +201,7 @@ uint32_t ModuleSymbolTable::getSymbolFlags(Symbol S)
const {
     Res |= BasicSymbolRef::SF_Executable;
   if (isa<GlobalAlias>(GV))
     Res |= BasicSymbolRef::SF_Indirect;
-  if (GV->hasPrivateLinkage())
+  if (GV->hasPrivateLinkage() || GV->hasAppendingLinkage())
     Res |= BasicSymbolRef::SF_FormatSpecific;
   if (!GV->hasLocalLinkage())
     Res |= BasicSymbolRef::SF_Global;

Is it a sane approach? Can it be upstreamed?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200220/47493955/attachment.html>

On 2020-02-20, Gleb Popov via llvm-dev wrote:
>On Tue, Feb 18, 2020 at 12:28 PM Gleb Popov <6yearold at gmail.com>
wrote:
>
>> Hello.
>>
>> I'm posting this question here, because there seem to be no
LLD-specific
>> mailing list. Sorry in advance if this is wrong one.
>>
>> I compile two C source with following command:
>>
>> clang -flto -o %name.bc %name.c
>>
>> LLVM is augmented with my custom pass, which amongst other things
create a
>> global with appending linkage:
>>
>> @myvar = appending constant [1 x [1 x i8]*] ...
>>
>> I also have another pass plugged into LTO pipeline, which turns this
>> global into internal one in the final module. I was hoping that LLD
would
>> first link bitcodes like llvm-link, appending @myvar's from both
modules
>> into a single one, then run my pass, and only then perform linking at
>> object level.
>>
>> However, LLD complains about duplicated symbol "myvar" and
doesn't even
>> run any passes.
>>
>> I've tracked the problem down to BitcodeFile::parse() function from
>>
https://github.com/llvm/llvm-project/blob/master/lld/COFF/InputFiles.cpp#L918
>> - it doesn't take linkage type into account.
>>
>> Can anything be done about this problem? Or was my approach broken from
>> the beginning?
>>
>> Thanks in advance.
>>
>
>
>The following patch has fixed the problem for me:
>
>diff --git a/llvm/lib/Object/ModuleSymbolTable.cpp
>b/llvm/lib/Object/ModuleSymbolTable.cpp
>index 33ce7d8109f..d1d74fa54bf 100644
>--- a/lib/Object/ModuleSymbolTable.cpp
>+++ b/lib/Object/ModuleSymbolTable.cpp
>@@ -201,7 +201,7 @@ uint32_t ModuleSymbolTable::getSymbolFlags(Symbol S)
>const {
>     Res |= BasicSymbolRef::SF_Executable;
>   if (isa<GlobalAlias>(GV))
>     Res |= BasicSymbolRef::SF_Indirect;
>-  if (GV->hasPrivateLinkage())
>+  if (GV->hasPrivateLinkage() || GV->hasAppendingLinkage())
>     Res |= BasicSymbolRef::SF_FormatSpecific;
>   if (!GV->hasLocalLinkage())
>     Res |= BasicSymbolRef::SF_Global;
>
>Is it a sane approach? Can it be upstreamed?
Can you paste the IR file for which you want AppendingLinkage to work?

 From an earlier command you pasted `clang -flto -o %name.bc %name.c`, it
seems that you have some way to generate `appending` from a .c file.