llvm dev - Jul 2014 - [LLVMdev] Global constructors "get lost" when transforming bitcode files

Hello,

A strange problem appears when upgrading from release_34 to testing. Some
transformations to bitcode files cause registered global_ctors to not be
called. Here's an example (I've also attached the complete example and
pasted it below):

This works:

    clang -fsanitize=address  -flto -c -o sum.o sum.c
    clang -fsanitize=address -o sum sum.o

This doesn't work:

    clang -fsanitize=address  -flto -c -o sum.o sum.c
    llvm-dis sum.o
    llvm-as sum.o.ll -o sum.o
    clang -fsanitize=address -o sum sum.o

The second version segfaults when accessing shadow memory, because the
memory has not been initialized, because __asan_init_* was never called.
This is surprising, because in the llvm-disassembly the global constructor
still shows up.

The llvm-dis/llvm-as operation should be a no-op, yet the global_ctors get
lost in the process. This happens also with other operations that affect
global_ctors, e.g., with "opt -insert-gcov-profiling".

The problem does not occur in the release_34 branch, but I have seen it on
both testing and master.

Any idea where this could come from would be much appreciated!
Jonas


testcase:

cat >sum.c <<EOF
#include <stdio.h>
#include <assert.h>

int main() {
    const int MAX_SIZE = 100;
    int a[MAX_SIZE];

    for (int i = 0; i < MAX_SIZE; ++i) {
        a[i] = i * i + 4;
    }

    int n_numbers;
    printf("How many numbers should I sum up? ");
    scanf("%d", &n_numbers);

    int sum = 0;
    for (int i = 0; i < n_numbers; ++i) {
        sum += a[i];
    }

    assert(sum >= 0);
    printf("The sum is: %d\n", sum);
    return 0;
}
EOF

set -ex
rm -f *.o *.ll sum

clang -fsanitize=address  -flto -c -o sum.o sum.c

llvm-dis sum.o
llvm-as sum.o.ll -o sum.o

clang -fsanitize=address -o sum sum.o

echo 22 | ./sum
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140703/4e250d66/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_ctors.sh
Type: application/x-sh
Size: 638 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140703/4e250d66/attachment.sh>

Hello,

I've narrowed the issue down to revision 209015: Add comdat key field to
llvm.global_ctors and llvm.global_dtors.

This revision introduces an additional field to global constructors which,
when non-null, can prevent constructors from being called. In my case, this
field is always set to null, yet global constructors are not called.

I've observed that this happens only on Mac, not on Linux. In the patch, I
haven't found anything OS-specific. However, the issue might be due to
different linker implementations.

Does anybody have additional insights into this? Should I file a bug report?

Best regards,
Jonas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140704/8b835547/attachment.html>

On Fri, Jul 4, 2014 at 10:55 AM, Jonas Wagner <jonas.wagner at epfl.ch>
wrote:
> Hello,
>
> I've narrowed the issue down to revision 209015: Add comdat key field
to
> llvm.global_ctors and llvm.global_dtors.
>
> This revision introduces an additional field to global constructors which,
> when non-null, can prevent constructors from being called. In my case, this
> field is always set to null, yet global constructors are not called.
>
> I've observed that this happens only on Mac, not on Linux. In the
patch, I
> haven't found anything OS-specific. However, the issue might be due to
> different linker implementations.
>
> Does anybody have additional insights into this? Should I file a bug
report?
>
Probably a good idea.

Rafael and David were looking at this sort of thing when I checked last...

-eric

Are you setting DYLD_LIBRARY_PATH when running on OS X? The linker on
OS X uses that to findl ibLTO.dylib.

At least for me setting DYLD_LIBRARY_PATH when running clang in the
linking stage fixes the problem.

I would still be curious to know how the old (system) libLTO.dylib can
read the bitcode output by clang but not llvm-as.


On 3 July 2014 13:06, Jonas Wagner <jonas.wagner at epfl.ch>
wrote:
> Hello,
>
> A strange problem appears when upgrading from release_34 to testing. Some
> transformations to bitcode files cause registered global_ctors to not be
> called. Here's an example (I've also attached the complete example
and
> pasted it below):
>
> This works:
>
>     clang -fsanitize=address  -flto -c -o sum.o sum.c
>     clang -fsanitize=address -o sum sum.o
>
> This doesn't work:
>
>     clang -fsanitize=address  -flto -c -o sum.o sum.c
>     llvm-dis sum.o
>     llvm-as sum.o.ll -o sum.o
>     clang -fsanitize=address -o sum sum.o
>
> The second version segfaults when accessing shadow memory, because the
> memory has not been initialized, because __asan_init_* was never called.
> This is surprising, because in the llvm-disassembly the global constructor
> still shows up.
>
> The llvm-dis/llvm-as operation should be a no-op, yet the global_ctors get
> lost in the process. This happens also with other operations that affect
> global_ctors, e.g., with "opt -insert-gcov-profiling".
>
> The problem does not occur in the release_34 branch, but I have seen it on
> both testing and master.
>
> Any idea where this could come from would be much appreciated!
> Jonas
>
>
> testcase:
>
> cat >sum.c <<EOF
> #include <stdio.h>
> #include <assert.h>
>
> int main() {
>     const int MAX_SIZE = 100;
>     int a[MAX_SIZE];
>
>     for (int i = 0; i < MAX_SIZE; ++i) {
>         a[i] = i * i + 4;
>     }
>
>     int n_numbers;
>     printf("How many numbers should I sum up? ");
>     scanf("%d", &n_numbers);
>
>     int sum = 0;
>     for (int i = 0; i < n_numbers; ++i) {
>         sum += a[i];
>     }
>
>     assert(sum >= 0);
>     printf("The sum is: %d\n", sum);
>     return 0;
> }
> EOF
>
> set -ex
> rm -f *.o *.ll sum
>
> clang -fsanitize=address  -flto -c -o sum.o sum.c
>
> llvm-dis sum.o
> llvm-as sum.o.ll -o sum.o
>
> clang -fsanitize=address -o sum sum.o
>
> echo 22 | ./sum
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

On Thu, Jul 3, 2014 at 10:06 AM, Jonas Wagner <jonas.wagner at epfl.ch>
wrote:
>
> The llvm-dis/llvm-as operation should be a no-op, yet the global_ctors get
> lost in the process. This happens also with other operations that affect
> global_ctors, e.g., with "opt -insert-gcov-profiling".
>
This isn't strictly true.  Deserializing old bitcode will auto-upgrade it.
 In this case, it will be auto-upgraded to the 3-field form of
llvm.global_ctors.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140707/ed87f1e9/attachment.html>

Hi,

Are you setting DYLD_LIBRARY_PATH when running on OS X? The linker
on
> OS X uses that to find libLTO.dylib.
>
This indeed solves the problem! Thanks for the hint. Although I had used
custom versions of Clang for a while, this is the first time I encountered
incompatibilities with the system linker, and so I wasn't aware of the need
to set DYLD_LIBRARY_PATH.

The llvm-dis/llvm-as operation should be a no-op, yet the global_ctors
get
>> lost in the process. This happens also with other operations that
affect
>> global_ctors, e.g., with "opt -insert-gcov-profiling".
>>
>
> This isn't strictly true.  Deserializing old bitcode will auto-upgrade
it.
>  In this case, it will be auto-upgraded to the 3-field form of
> llvm.global_ctors.

I see. I wonder if this is what's happening. Clang and llvm-as are built
against the same version of LLVM, so Clang should not emit old bitcode. On
the other hand, the fact that the system linker understands Clang-generated
bitcode probably means it used the old format.

Best,
Jonas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140710/fc111430/attachment.html>