llvm dev - Jan 2019 - Difference when compiling human readable IR vs bitcode with clang frontend

We've noticed a difference in the embedded bitcode when compiling human
readable IR to an object directly vs first compiling IR to BC and then an object
through clang -cc1.
If the original IR file contained an "llvm.compiler.used" gv, it will
be preserved when compiling IR -> BC -> Obj.
When compiling IR -> Obj directly, it will be removed.

This difference does not exist for the "llvm.used" gv however, it is
always preserved.
This questions seems related to the following lit test in LLVM:
https://github.com/llvm-mirror/llvm/blob/master/test/Transforms/GlobalOpt/compiler-used.ll.

Is this somehow expected behaviour?

Reproduce:
Source taken from the lit test.

define void @foo() {
  ret void
}

@llvm.used = appending global [1 x i8*] [i8* bitcast (void ()* @foo to i8*)],
section "llvm.metadata"
@llvm.compiler.used = appending global [1 x i8*] [i8* bitcast (void ()* @foo to
i8*)], section "llvm.metadata"


# Compile IR -> Obj directly.
clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-obj -fembed-bitcode=all -x
ir test.ll -o test_ll.o

# Compile IR -> BC -> Obj.
clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-llvm-bc -fblocks
-fencode-extended-block-signature -x ir test.ll  -o test.bc
clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-obj -fembed-bitcode=all -x
ir test.bc -o test_bc.o

# Extract and disassemble embedded bitcode from both scenarios.
segedit test_bc.o -extract __LLVM __bitcode bc_bc.bc
segedit test_ll.o -extract __LLVM __bitcode ll_bc.bc
llvm-dis bc_bc.bc
llvm-dis ll_bc.bc

# Diff both IR files to show that only bc_bc.ll contains
"llvm.compiler.used"
diff bc_bc.ll ll_bc.ll

- Dennis Frett
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1389 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190118/09565627/attachment.bin>

> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of
> Dennis Frett via llvm-dev
> Sent: Friday, January 18, 2019 4:15 AM
> To: llvm-dev at lists.llvm.org
> Subject: [llvm-dev] Difference when compiling human readable IR vs bitcode
> with clang frontend
> 
> We've noticed a difference in the embedded bitcode when compiling human
> readable IR to an object directly vs first compiling IR to BC and then an
> object through clang -cc1.
> If the original IR file contained an "llvm.compiler.used" gv, it
will be
> preserved when compiling IR -> BC -> Obj.
> When compiling IR -> Obj directly, it will be removed.
> 
> This difference does not exist for the "llvm.used" gv however, it
is
> always preserved.
> This questions seems related to the following lit test in LLVM:
> https://github.com/llvm-
> mirror/llvm/blob/master/test/Transforms/GlobalOpt/compiler-used.ll.
> 
> Is this somehow expected behaviour?
I am curious what happens if you do IR -> BC -> IR -> BC; I'd
expect
the IR to more-or-less match (the differences all being due to one
is hand-written and one is a disassembly) and the two BC files should
be identical.

If not (and I'm am guessing they aren't, which is why you see some
differences in the compiled object file) that's a bug.
--paulr
> 
> Reproduce:
> Source taken from the lit test.
> 
> define void @foo() {
>   ret void
> }
> 
> @llvm.used = appending global [1 x i8*] [i8* bitcast (void ()* @foo to
> i8*)], section "llvm.metadata"
> @llvm.compiler.used = appending global [1 x i8*] [i8* bitcast (void ()*
> @foo to i8*)], section "llvm.metadata"
> 
> 
> # Compile IR -> Obj directly.
> clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-obj -fembed-
> bitcode=all -x ir test.ll -o test_ll.o
> 
> # Compile IR -> BC -> Obj.
> clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-llvm-bc -fblocks -
> fencode-extended-block-signature -x ir test.ll  -o test.bc
> clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-obj -fembed-
> bitcode=all -x ir test.bc -o test_bc.o
> 
> # Extract and disassemble embedded bitcode from both scenarios.
> segedit test_bc.o -extract __LLVM __bitcode bc_bc.bc
> segedit test_ll.o -extract __LLVM __bitcode ll_bc.bc
> llvm-dis bc_bc.bc
> llvm-dis ll_bc.bc
> 
> # Diff both IR files to show that only bc_bc.ll contains
> "llvm.compiler.used"
> diff bc_bc.ll ll_bc.ll
> 
> - Dennis Frett

Going from IR <-> BC does not seem to create a difference.
IR -> BC -> IR -> BC, either with clang frontent or by using llvm-as
and llvm-dis yields identical bc files.

I have only been able to reproduce this issue when emitting to an object file.

> On 18 Jan 2019, at 18:09, <paul.robinson at sony.com>
<paul.robinson at sony.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of
>> Dennis Frett via llvm-dev
>> Sent: Friday, January 18, 2019 4:15 AM
>> To: llvm-dev at lists.llvm.org
>> Subject: [llvm-dev] Difference when compiling human readable IR vs
bitcode
>> with clang frontend
>> 
>> We've noticed a difference in the embedded bitcode when compiling
human
>> readable IR to an object directly vs first compiling IR to BC and then
an
>> object through clang -cc1.
>> If the original IR file contained an "llvm.compiler.used" gv,
it will be
>> preserved when compiling IR -> BC -> Obj.
>> When compiling IR -> Obj directly, it will be removed.
>> 
>> This difference does not exist for the "llvm.used" gv
however, it is
>> always preserved.
>> This questions seems related to the following lit test in LLVM:
>> https://github.com/llvm-
>> mirror/llvm/blob/master/test/Transforms/GlobalOpt/compiler-used.ll.
>> 
>> Is this somehow expected behaviour?
> 
> I am curious what happens if you do IR -> BC -> IR -> BC; I'd
expect
> the IR to more-or-less match (the differences all being due to one
> is hand-written and one is a disassembly) and the two BC files should
> be identical.
> 
> If not (and I'm am guessing they aren't, which is why you see some
> differences in the compiled object file) that's a bug.
> --paulr
> 
>> 
>> Reproduce:
>> Source taken from the lit test.
>> 
>> define void @foo() {
>>  ret void
>> }
>> 
>> @llvm.used = appending global [1 x i8*] [i8* bitcast (void ()* @foo to
>> i8*)], section "llvm.metadata"
>> @llvm.compiler.used = appending global [1 x i8*] [i8* bitcast (void ()*
>> @foo to i8*)], section "llvm.metadata"
>> 
>> 
>> # Compile IR -> Obj directly.
>> clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-obj -fembed-
>> bitcode=all -x ir test.ll -o test_ll.o
>> 
>> # Compile IR -> BC -> Obj.
>> clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-llvm-bc -fblocks -
>> fencode-extended-block-signature -x ir test.ll  -o test.bc
>> clang -cc1 -triple x86_64-apple-macosx10.13.0 -emit-obj -fembed-
>> bitcode=all -x ir test.bc -o test_bc.o
>> 
>> # Extract and disassemble embedded bitcode from both scenarios.
>> segedit test_bc.o -extract __LLVM __bitcode bc_bc.bc
>> segedit test_ll.o -extract __LLVM __bitcode ll_bc.bc
>> llvm-dis bc_bc.bc
>> llvm-dis ll_bc.bc
>> 
>> # Diff both IR files to show that only bc_bc.ll contains
>> "llvm.compiler.used"
>> diff bc_bc.ll ll_bc.ll
>> 
>> - Dennis Frett
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190120/537bc139/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1389 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190120/537bc139/attachment.bin>