llvm dev - Apr 2020 - How to optimize a large project in llvm pass?

Hi,
I have written MyPass.cpp and I use "EP_FullLinkTimeOptimizationLast".
I compile it and get libMypass.so.
MyPass.so rely on some other code, such as，Init.c and Helper.c.
So, I  use following commands to get a static library libmypass-rt.a.
"
$ clang -fsanitize=safe-stack -c -emit-llvm -fpic -Wall -Wextra -O2 -I
../include -o obj/helpers.o helpers.c
$ clang -fsanitize=safe-stack -c -emit-llvm -fpic -Wall -Wextra -O2 -I
../include -o obj/init.o init.c
$ ar rcs obj/libmypass-rt.a obj/helpers.o obj/init.o
."
When mypass optimizes the target program, it reles on libmypass-rt.a
library. So I need to link libmypass-rt.a to the target program, then
optimize after link.
So, I need LTO.
I use following commands:
$ export CC=clang
$ export CXX=clang++
$ export RANLIB=llvm-ranlib
$ export  LIBS="path/to/.../libmypass-rt.a"
$ export CFLAGS=" -fsanitize=safe-stack -Wall -Wextra -fpic -flto
-Xclang -load -Xclang path/to/.../libmypass.so"
$ export LDFLAGS=" -flto -fuse-ld=gold"
$ ./configure
$ make
But, errors occurred. I check out config.log
/usr/bin/ld.gold: error: path/to/.../libmemsentry-rt.a
: no archive symbol table (run ranlib)
clang-10: error: linker command failed with exit code 1
 (use -v to see invocation)

What shuold I do?
If this problem is solved, please tell me whether my method can achieve my
goal？

Thansk!



   -
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200420/c240d4c9/attachment.html>

Hi,

` -Xclang -load -Xclang path/to/.../libmypass.so` is gonna load the pass
inside clang, however clang will invoke the linker separately and it won't
have your pass. I don't know if gold or lld can load dynamically passes
like this?
An alternative it to link your pass statically with the gold plugin or with
lld.

Best,

-- 
Mehdi


On Sun, Apr 19, 2020 at 11:53 PM y liu via llvm-dev <llvm-dev at
lists.llvm.org>
wrote:
> Hi,
> I have written MyPass.cpp and I use
"EP_FullLinkTimeOptimizationLast".
> I compile it and get libMypass.so.
> MyPass.so rely on some other code, such as，Init.c and Helper.c.
> So, I  use following commands to get a static library libmypass-rt.a.
> "
> $ clang -fsanitize=safe-stack -c -emit-llvm -fpic -Wall -Wextra -O2 -I
> ../include -o obj/helpers.o helpers.c
> $ clang -fsanitize=safe-stack -c -emit-llvm -fpic -Wall -Wextra -O2 -I
> ../include -o obj/init.o init.c
> $ ar rcs obj/libmypass-rt.a obj/helpers.o obj/init.o
> ."
> When mypass optimizes the target program, it reles on libmypass-rt.a
> library. So I need to link libmypass-rt.a to the target program, then
> optimize after link.
> So, I need LTO.
> I use following commands:
> $ export CC=clang
> $ export CXX=clang++
> $ export RANLIB=llvm-ranlib
> $ export  LIBS="path/to/.../libmypass-rt.a"
>
> $ export CFLAGS=" -fsanitize=safe-stack -Wall -Wextra -fpic -flto
-Xclang -load -Xclang path/to/.../libmypass.so"
> $ export LDFLAGS=" -flto -fuse-ld=gold"
> $ ./configure
> $ make
> But, errors occurred. I check out config.log
> /usr/bin/ld.gold: error: path/to/.../libmemsentry-rt.a
> : no archive symbol table (run ranlib)
> clang-10: error: linker command failed with exit code 1
>  (use -v to see invocation)
>
> What shuold I do?
> If this problem is solved, please tell me whether my method can achieve my
> goal？
>
> Thansk!
>
>
>
>    -
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200420/f50add3a/attachment.html>

To me it looks like you may just want to link your output binary with 
libmypass-rt.a. Is that correct? For that you only need to pass 
libmypass-rt.a on the command line when linking (as you would any other 
library that you want linked in). This is really well explained in this 
blog by Adrian Sampson: https://www.cs.cornell.edu/~asampson/blog/llvm.html

@Mehdi Re loading plugins in LLD: https://reviews.llvm.org/D75879

-Andrzej

On 20/04/2020 16:31, Mehdi AMINI via llvm-dev wrote:
> Hi,
> 
> ` -Xclang -load -Xclang path/to/.../libmypass.so` is gonna load the pass 
> inside clang, however clang will invoke the linker separately and it 
> won't have your pass. I don't know if gold or lld can load
dynamically
> passes like this?
> An alternative it to link your pass statically with the gold plugin or 
> with lld.
> 
> Best,
> 
> -- 
> Mehdi
> 
> 
> On Sun, Apr 19, 2020 at 11:53 PM y liu via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
> 
>     Hi,
>     I have written MyPass.cpp and I use
"EP_FullLinkTimeOptimizationLast".
>     I compile it and get libMypass.so.
>     MyPass.so rely on some other code, such as，Init.c and Helper.c.
>     So, I  use following commands to get a static library libmypass-rt.a.
>     "
>     $ clang -fsanitize=safe-stack -c -emit-llvm -fpic -Wall -Wextra -O2
>     -I ../include -o obj/helpers.o helpers.c
>     $ clang -fsanitize=safe-stack -c -emit-llvm -fpic -Wall -Wextra -O2
>     -I ../include -o obj/init.o init.c
>     $ ar rcs obj/libmypass-rt.a obj/helpers.o obj/init.o
>     ."
>     When mypass optimizes the target program, it reles on libmypass-rt.a
>     library. So I need to link libmypass-rt.a to the target program,
>     then optimize after link.
>     So, I need LTO.
>     I use following commands:
>     $ export CC=clang
>     $ export CXX=clang++
>     $ export RANLIB=llvm-ranlib
>     $ export  LIBS="path/to/.../libmypass-rt.a"
>    
$ export CFLAGS=" -fsanitize=safe-stack -Wall -Wextra -fpic -flto -Xclang -load -Xclang path/to/.../libmypass.so"
>     $ export LDFLAGS=" -flto -fuse-ld=gold"
>     $ ./configure
>     $ make
>     But, errors occurred. I check out config.log
>     /usr/bin/ld.gold:
>    
error: path/to/.../libmemsentry-rt.a: no archive symbol table (run ranlib)
>     clang-10: error: linker command failed with exit code
>     1 (use -v to see invocation)
> 
>     What shuold I do?
>     If this problem is solved, please tell me whether my method can
>     achieve my goal？
> 
>     Thansk!
> 
> 
>       *
> 
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>