llvm dev - Mar 2020 - llvm-link has no --allow-multiple-definition

Hello,

I am developing an LTO instrumentation of programs for fuzzing (for
afl++) and I run into the following problem:

a lot of open source program have multiple definitons of the same
functions. In normal compile mode this is not a problem, however with
LTO this only works with the linker option:
 -Wl,--allow-multiple-definition

Now to be able to instrument (it is an "opt" pass") at link time
I need
to merge all bitcode files together, and this works great - unless I run
into this error when I link them to llvm-link:

error: Linking globals named 'process_arg': symbol multiply defined!

(this is for the bogofilter tools, but happens with other open source
program too).

My issue is that neither is --allow-multiple-definition an llvm-link
command line option, nor can I find an equivalent from the --help output.

is there a plan to add this (IMHO) important feature? Thank you!

Regards,
Marc

-- 
Marc Heuse
www.mh-sec.de

PGP: AF3D 1D4C D810 F0BB 977D  3807 C7EE D0A0 6BE9 F573

Despite the name, llvm-link isn't really a proper linker.  If you need
symbols to get resolved exactly the same way an ELF linker would resolve them, I
would recommend using lld.

It's possible to write a pass plugin that hooks into the LTO pipeline. 
http://llvm.org/docs/WritingAnLLVMPass.html describes this.

-Eli
> -----Original Message-----
> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Marc
via llvm-
> dev
> Sent: Sunday, March 8, 2020 10:01 AM
> To: llvm-dev <llvm-dev at lists.llvm.org>
> Subject: [EXT] [llvm-dev] llvm-link has no --allow-multiple-definition
>
> Hello,
>
> I am developing an LTO instrumentation of programs for fuzzing (for
> afl++) and I run into the following problem:
>
> a lot of open source program have multiple definitons of the same
> functions. In normal compile mode this is not a problem, however with
> LTO this only works with the linker option:
>  -Wl,--allow-multiple-definition
>
> Now to be able to instrument (it is an "opt" pass") at link
time I need
> to merge all bitcode files together, and this works great - unless I run
> into this error when I link them to llvm-link:
>
> error: Linking globals named 'process_arg': symbol multiply
defined!
>
> (this is for the bogofilter tools, but happens with other open source
> program too).
>
> My issue is that neither is --allow-multiple-definition an llvm-link
> command line option, nor can I find an equivalent from the --help output.
>
> is there a plan to add this (IMHO) important feature? Thank you!
>
> Regards,
> Marc
>
> --
> Marc Heuse
> www.mh-sec.de
>
> PGP: AF3D 1D4C D810 F0BB 977D  3807 C7EE D0A0 6BE9 F573
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi Eli,

I know the page, and the only mentioning about LTO or link time
optimization I found is the option
PassManagerBuilder::EP_FullLinkTimeOptimizationLast - and nobody knows
how this works (and asked about it on this list too before with no
reply), and the author who wrote that part in the llvm code at Redhat
did not reply to a question how to actually use it. And there is no
example code or any code using this option anywhere on the Internet I
could find.

And loading such a plugin via -Xclang -load -Xclang plugin.so with
EP_FullLinkTimeOptimizationLast doesnt work.

So if you can shed any light about how to write a plugin that allows
running a pass within ld - I would be very greatful! (and a large
community of people fuzzing code :)

Regards,
Marc


On 09.03.20 20:24, Eli Friedman wrote:
> Despite the name, llvm-link isn't really a proper linker.  If you need
symbols to get resolved exactly the same way an ELF linker would resolve them, I
would recommend using lld.
> 
> It's possible to write a pass plugin that hooks into the LTO pipeline. 
http://llvm.org/docs/WritingAnLLVMPass.html describes this.
> 
> -Eli
> 
>> -----Original Message-----
>> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of
Marc via llvm-
>> dev
>> Sent: Sunday, March 8, 2020 10:01 AM
>> To: llvm-dev <llvm-dev at lists.llvm.org>
>> Subject: [EXT] [llvm-dev] llvm-link has no --allow-multiple-definition
>>
>> Hello,
>>
>> I am developing an LTO instrumentation of programs for fuzzing (for
>> afl++) and I run into the following problem:
>>
>> a lot of open source program have multiple definitons of the same
>> functions. In normal compile mode this is not a problem, however with
>> LTO this only works with the linker option:
>>  -Wl,--allow-multiple-definition
>>
>> Now to be able to instrument (it is an "opt" pass") at
link time I need
>> to merge all bitcode files together, and this works great - unless I
run
>> into this error when I link them to llvm-link:
>>
>> error: Linking globals named 'process_arg': symbol multiply
defined!
>>
>> (this is for the bogofilter tools, but happens with other open source
>> program too).
>>
>> My issue is that neither is --allow-multiple-definition an llvm-link
>> command line option, nor can I find an equivalent from the --help
output.
>>
>> is there a plan to add this (IMHO) important feature? Thank you!
>>
>> Regards,
>> Marc
>>
>> --
>> Marc Heuse
>> www.mh-sec.de
>>
>> PGP: AF3D 1D4C D810 F0BB 977D  3807 C7EE D0A0 6BE9 F573
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 
Marc Heuse
Mobil: +49 177 9611560
Fax: +49 30 37309726
www.mh-sec.de

Marc Heuse - IT-Security Consulting
Winsstr. 68
10405 Berlin

Ust.-Ident.-Nr.: DE244222388
PGP: AF3D 1D4C D810 F0BB 977D  3807 C7EE D0A0 6BE9 F573