llvm dev - May 2016 - How to debug if LTO generate wrong code?

Hi Umesh,

Thank you for the suggestion. I can use the "Brute force method " to
narrow down the LTO wrong instructions here and there, but I still don't
know why these wrong instructions are generated, and how to let Clang LTO
don't generate those wrong instructions.

I suspect the wrong code is caused by some LTO wrong optimization pass, so I
hope to disable all optimizations in the LTO firstly, then enable them one by
one later to narrow down my issue root cause. But when I try to disable the
optimization by enforcing –O0 in the LTO build, I find the ld fails to recognize
some clang  bitcode library, and fail to link.



e.g. use the Clang_LTO_Fails_On_LD example in below bug attachment

https://sourceware.org/bugzilla/show_bug.cgi?id=20070



If I enforce the -O0 to disable the optimization in LTO, the ld fail to link:

~/clang38/bin/clang -o Hello.dll -flto -O0 -nostdlib -Wl,-n -Wl,-q
-Wl,--gc-sections -Wl,-z,common-page-size=0x40 -Wl,--entry,_ModuleEntryPoint
-Wl,-u,_ModuleEntryPoint -Wl,-Map,Hello.map -Wl,-melf_x86_64
-Wl,--oformat=elf64-x86-64 -Wl,--start-group,, at static_library_files.lst
-Wl,--end-group

BaseLib.lib: error adding symbols: File format not recognized

clang-3.8: error: linker command failed with exit code 1 (use -v to see
invocation)



But if I enable the -O1, -O2, or higher -On, the ld  link pass:

~/clang38/bin/clang -o Hello.dll -flto -O1 -nostdlib -Wl,-n -Wl,-q
-Wl,--gc-sections -Wl,-z,common-page-size=0x40 -Wl,--entry,_ModuleEntryPoint
-Wl,-u,_ModuleEntryPoint -Wl,-Map,Hello.map -Wl,-melf_x86_64
-Wl,--oformat=elf64-x86-64 -Wl,--start-group,, at static_library_files.lst
-Wl,--end-group





How can I correctly disable all the optimization in clang LTO? How can I know,
enable and disable the specific optimizations in clang LTO? Any suggestion is
welcomed!







Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522


> -----Original Message-----
> From: Umesh Kalappa [mailto:umesh.kalappa0 at gmail.com]
> Sent: Saturday, May 14, 2016 2:14 AM
> To: Shi, Steven <steven.shi at intel.com>
> Cc: llvm-dev <llvm-dev at lists.llvm.org>; cfe-dev at lists.llvm.org
> Subject: Re: [llvm-dev] How to debug if LTO generate wrong code?
>
> Steven,
>
> Brute force method  is ,get the disassemble of the hanged function and
> try to check the difference with and without LTO in the generated
> code.
>
> or try to attach gdb and check  for the instruction ,that cause the
exception .
>
> Thank you
> ~Umesh
>
> On Fri, May 13, 2016 at 7:48 PM, Shi, Steven via llvm-dev
> <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
wrote:
> > Hello,
> >
> > I'm enabling clang LTO to improve code size of Uefi standard
> > (http://www.uefi.org/) firmware (https://github.com/tianocore/edk2),
> which
> > is mostly C code. My project is in https://github.com/shijunjing/edk2
> branch
> > llvm : https://github.com/shijunjing/edk2/tree/llvm. I find my most
> firmware
> > modules work well after enable LTO, but some X64 modules will not run
> (e.g.
> > hang with CPU exception) , and these X64 modules work well if build
with
> the
> > LTO disabled (-fno-lto).
> >
> > I don’t know how to efficiently debug these LTO wrong code and
> investigate
> > if there is compiler’s bug. I appreciate if anyone can  give me some
> > suggestions about the clang LTO issue debug method, commands, or BKMs.
> >
> >
> >
> > Below are  my clang LTO build tools and options, I use clang 3.8
release
> > with binutils 2.26 ld (I’ve pushed ld support LLVM gold plugin
> > https://sourceware.org/bugzilla/show_bug.cgi?id=20070). Any suggestion
> is
> > welcome!
> >
> >
> >
> >
> >
> >
> >
> > ##################
> >
> > # CLANGLTO38 X64 definitions
> >
> > ##################
> >
> > *_CLANGLTO38_X64_OBJCOPY_PATH         
> DEF(GCC53_X64_PREFIX)objcopy
> >
> > *_CLANGLTO38_X64_CC_PATH              = DEF(CLANG38_X64_PREFIX)clang
> >
> > *_CLANGLTO38_X64_SLINK_PATH           = DEF(CLANG38_X64_PREFIX)llvm-
> ar
> >
> > *_CLANGLTO38_X64_DLINK_PATH           
> DEF(CLANG38_X64_PREFIX)clang
> >
> > *_CLANGLTO38_X64_ASM_PATH             = DEF(CLANG38_X64_PREFIX)clang
> >
> > *_CLANGLTO38_X64_PP_PATH              = DEF(CLANG38_X64_PREFIX)clang
> >
> > *_CLANGLTO38_X64_RC_PATH              = DEF(GCC53_X64_PREFIX)objcopy
> >
> >
> >
> > *_CLANGLTO38_X64_CC_FLAGS = -c -fshort-wchar -fno-strict-aliasing -
> Wall
> > -Werror -Wno-array-bounds -Wno-empty-body -ffunction-sections
> > -fdata-sections -include AutoGen.h -
> DSTRING_ARRAY_NAME=$(BASE_NAME)Strings
> > -fno-stack-protector -fno-builtin -mms-bitfields -Wno-address
> > -Wno-shift-negative-value -Wno-parentheses-equality -Wno-unknown-
> pragmas
> > -Wno-tautological-constant-out-of-range-compare
> > -Wno-incompatible-library-redeclaration -target x86_64-pc-linux-gnu
> > -fno-asynchronous-unwind-tables -m64 -Wno-enum-conversion
> > "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone
-mcmodel=large -g -Os
> -flto
> >
> > *_CLANGLTO38_X64_DLINK_FLAGS  = -flto -nostdlib -Wl,-n -Wl,-q
> > -Wl,--gc-sections -Wl,-z,common-page-size=0x40
> > -Wl,--entry,$(IMAGE_ENTRY_POINT) -Wl,-u,$(IMAGE_ENTRY_POINT)
> > -Wl,-Map,$(DEST_DIR_DEBUG)/$(BASE_NAME).map -Wl,-melf_x86_64
> > -Wl,--oformat=elf64-x86-64
> >
> > *_CLANGLTO38_X64_ASM_FLAGS            = -c -x assembler -imacros
> > $(DEST_DIR_DEBUG)/AutoGen.h -m64 -target x86_64-pc-linux-gnu
> >
> > *_CLANGLTO38_X64_RC_FLAGS             = -I binary -O elf64-x86-64     
-B
> > i386    --rename-section .data=.hii
> >
> > *_CLANGLTO38_X64_NASM_FLAGS           = -f elf64
> >
> >
> >
> >
> >
> >
> >
> > Steven Shi
> >
> > Intel\SSG\STO\UEFI Firmware
> >
> >
> >
> > Tel: +86 021-61166522
> >
> > iNet: 821-6522
> >
> >
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160516/540e803b/attachment.html>

Hello,
Let me ask a LTO simple question again. For the llvm LTO example in the link:
http://llvm.org/docs/LinkTimeOptimization.html, I use below build commands to
generate three different optimization level binary: -O0, -O1, -O2. By nm listing
the foo1~4 symbols , I can see these different optimizations really works.

1.       How can I know what different optimizations are used by the clang LTO
among -O0, -O1 and -O2?

2.       Is the compiler domain optimization (e.g. clang/llvm) or the linker
(e.g. ld) domain optimization make these difference?

3.       How can I explicitly enable or disable these specific optimizations
besides using -O0, -O1, -O2?


$clang -emit-llvm -c main.c -o main.bc
$clang -emit-llvm -c a.c -o a.bc
$llvm-ar cr main.lib main.bc
$llvm-ar cr a.lib a.bc
$clang -O0 -flto main.lib a.lib -o main0
$clang -O1 -flto main.lib a.lib -o main1
$clang -O2 -flto main.lib a.lib -o main2

$nm main0
…
00000000004005a0 t foo1
0000000000400580 t foo2
00000000004005e0 t foo3
0000000000400530 t foo4
0000000000400500 t frame_dummy
…
$ nm main1
…
0000000000400550 t foo1
0000000000400580 t foo3
0000000000400530 t foo4
0000000000400500 t frame_dummy
…
$ nm main2
…
00000000004004d0 t frame_dummy
…

From blew verbose output, tt seems only linker( e.g. ld) is invovled to do the
optimization?

$ clang -O2 -flto main.lib a.lib -o main2 -v
clang version 3.8.0 (tags/RELEASE_380/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.3
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.3.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0
Found candidate GCC installation:
/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0
Selected GCC installation: /usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
"/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m
elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o main2
/usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o
/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/crtbegin.o
-L/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0
-L/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/../../../../lib64
-L/usr/local/bin/../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64
-L/usr/lib/x86_64-linux-gnu
-L/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/../../..
-L/usr/local/bin/../lib -L/lib -L/usr/lib -plugin
/usr/local/bin/../lib/LLVMgold.so -plugin-opt=mcpu=x86-64 -plugin-opt=O2
main.lib a.lib -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed
-lgcc_s --no-as-needed
/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/crtend.o
/usr/lib/x86_64-linux-gnu/crtn.o


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160517/7b22b9d1/attachment.html>

> On May 17, 2016, at 1:33 AM, Shi, Steven via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hello,
> Let me ask a LTO simple question again. For the llvm LTO example in the
link:http://llvm.org/docs/LinkTimeOptimization.html
<http://llvm.org/docs/LinkTimeOptimization.html>, I use below build
commands to generate three different optimization level binary: -O0, -O1, -O2.
By nm listing the foo1~4 symbols , I can see these different optimizations
really works.
> 1.       How can I know what different optimizations are used by the clang
LTO among -O0, -O1 and -O2?
LTO is linker specific, clang is only forwarding the option to the linker here.
> 2.       Is the compiler domain optimization (e.g. clang/llvm) or the
linker (e.g. ld) domain optimization make these difference?
In you case, you invoke clang with "emit-llvm", without any
optimization level, so you get O0.
For what the linker is doing at these optimizations levels, again this is linker
specific.
> 3.       How can I explicitly enable or disable these specific
optimizations besides using -O0, -O1, -O2?
If you're talking about the LTO, this is linker specific again (ld is not
the same program on every platform). For instance there is no such thing as
O0/O1/O2 on OS X.

>  
>  
> $clang -emit-llvm -c main.c -o main.bc
> $clang -emit-llvm -c a.c -o a.bc
> $llvm-ar cr main.lib main.bc
> $llvm-ar cr a.lib a.bc
> $clang -O0 -flto main.lib a.lib -o main0
> $clang -O1 -flto main.lib a.lib -o main1
> $clang -O2 -flto main.lib a.lib -o main2
>  
> $nm main0
> …
> 00000000004005a0 t foo1
> 0000000000400580 t foo2
> 00000000004005e0 t foo3
> 0000000000400530 t foo4
> 0000000000400500 t frame_dummy
> …
> $ nm main1
> …
> 0000000000400550 t foo1
> 0000000000400580 t foo3
> 0000000000400530 t foo4
> 0000000000400500 t frame_dummy
> …
> $ nm main2
> …
> 00000000004004d0 t frame_dummy
> …
>  
> From blew verbose output, tt seems only linker( e.g. ld) is invovled to do
the optimization?
Yes.
Usually the LTO pipeline is a bit different from what you're doing, I'm
used to see:

$clang -flto -O3 -c main.c -o main.o
$clang -flto -O3 -c a.c -o a.o
$clang -flto -O3 main.o a.o -o main0


-- 
Mehdi


>  
> $ clang -O2 -flto main.lib a.lib -o main2 -v
> clang version 3.8.0 (tags/RELEASE_380/final)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
> InstalledDir: /usr/local/bin
> Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
> Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.3
> Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.3.1
> Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0
> Found candidate GCC installation:
/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0
> Selected GCC installation:
/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0
> Candidate multilib: .;@m64
> Candidate multilib: 32;@m32
> Selected multilib: .;@m64
> "/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr
-m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o main2
/usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o
/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/crtbegin.o
-L/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0
-L/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/../../../../lib64
-L/usr/local/bin/../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64
-L/usr/lib/x86_64-linux-gnu
-L/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/../../..
-L/usr/local/bin/../lib -L/lib -L/usr/lib -plugin
/usr/local/bin/../lib/LLVMgold.so -plugin-opt=mcpu=x86-64 -plugin-opt=O2
main.lib a.lib -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed
-lgcc_s --no-as-needed
/usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.0.0/crtend.o
/usr/lib/x86_64-linux-gnu/crtn.o
>  
>  
> Steven Shi
> Intel\SSG\STO\UEFI Firmware
>  
> Tel: +86 021-61166522
> iNet: 821-6522
>  
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160517/2cc421f2/attachment.html>