thr3ads.net - llvm dev - [LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests [Jan 2013]

If this information is useful, please help other people find it:
Share via:

Alexey Samsonov

2013-Jan-29 11:36 UTC

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

Hi!

I'm trying to run LLVM test suite under AddressSanitizer and get test
failures in:
    LLVM :: ExecutionEngine/MCJIT/simpletest-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-data-align-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-fp-no-external-funcs-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-global-init-nonzero-remote.ll

All of them fail with assertion:
lli:
/usr/local/google/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp:230:
void llvm::RuntimeDyldELF::resolveX86_64Relocation(const llvm::SectionEntry
&, uint64_t, uint64_t, uint32_t, int64_t): Assertion `RealOffset
<(2147483647) && RealOffset >= (-2147483647-1)' failed.

The reason is that AddressSanitizer replaces system malloc with its own
allocator, which
allocates memory at "unusual" parts of heap and the difference between
pointers can be significant
(and doesn't fit in 32 bytes).

I add debug output to calculation of RealOffset in resolveX86_64Relocation:

    uint64_t  FinalAddress = Section.LoadAddress + Offset;
    int64_t RealOffset = *Placeholder + Value + Addend - FinalAddress;
    fprintf(stderr, "%x + %lx + %lx - %lx = %lx\n",
            *Placeholder, Value, Addend, FinalAddress, RealOffset);
    assert(RealOffset <= INT32_MAX && RealOffset >= INT32_MIN);

This is what I get for system malloc:

0 + 7fec9867a000 + 0 - 7fec9867a040 = ffffffffffffffc0

This is what I get for ASan allocator (which results in assert failure):

0 + 600c0000a8a0 + 0 - 6018000090a0 = fffffff400001800


--
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130129/7557d316/attachment.html>

Adhemerval Zanella

2013-Jan-29 12:47 UTC

head link

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

I'm not quite familiar with x86_64 ABI but I faced a similar issue while
coding the PPC64
MCJIT support: some functions calls could not be referenced in 24-bits branch
offset.
I believe you'll need to handle it at
'RuntimeDyldELF::processRelocationRef' (look on how
ARM, MIPS or PPC64 do) and possible create a stub function.

On 01/29/2013 09:36 AM, Alexey Samsonov wrote:> Hi!
>
> I'm trying to run LLVM test suite under AddressSanitizer and get test
failures in:
>     LLVM :: ExecutionEngine/MCJIT/simpletest-remote.ll
>     LLVM :: ExecutionEngine/MCJIT/test-data-align-remote.ll
>     LLVM :: ExecutionEngine/MCJIT/test-fp-no-external-funcs-remote.ll
>     LLVM :: ExecutionEngine/MCJIT/test-global-init-nonzero-remote.ll
>
> All of them fail with assertion:
> lli:
/usr/local/google/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp:230:
void llvm::RuntimeDyldELF::resolveX86_64Relocation(const llvm::SectionEntry
&, uint64_t, uint64_t, uint32_t, int64_t): Assertion `RealOffset <=
(2147483647) && RealOffset >= (-2147483647-1)' failed.
>
> The reason is that AddressSanitizer replaces system malloc with its own
allocator, which
> allocates memory at "unusual" parts of heap and the difference
between pointers can be significant
> (and doesn't fit in 32 bytes).
>
> I add debug output to calculation of RealOffset in resolveX86_64Relocation:
>
>     uint64_t  FinalAddress = Section.LoadAddress + Offset;
>     int64_t RealOffset = *Placeholder + Value + Addend - FinalAddress;
>     fprintf(stderr, "%x + %lx + %lx - %lx = %lx\n",
>             *Placeholder, Value, Addend, FinalAddress, RealOffset);
>     assert(RealOffset <= INT32_MAX && RealOffset >=
INT32_MIN);
>
> This is what I get for system malloc:
> 0 + 7fec9867a000 + 0 - 7fec9867a040 = ffffffffffffffc0
> This is what I get for ASan allocator (which results in assert failure):
> 0 + 600c0000a8a0 + 0 - 6018000090a0 = fffffff400001800
>
> --
> Alexey Samsonov, MSK
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Kaylor, Andrew

2013-Jan-29 17:40 UTC

head link

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

Hi Alexey,

I think the most likely way to resolve this is to have the
RecordingMemoryManager do something more complex to manage its allocations in
such a way as to guarantee that they are all within proximity of one another. 
The code that is asserting is handling a relocation where code was generated to
use a 32-bit relative offset in 64-bit code.  If the two sections involved
really are more than a 32-bit offset apart then the generated code will not
work.

Alternatively, we could have MCJIT use whatever code generation option will
prevent 32-bit relative relocations from being generated in the first place. 
That would probably be preferable, but I haven't had success trying to do
that in limited efforts up to now.

As it happens, I'm working with the '-use-remote' option for lli
this week trying to add support for actual out-of-process execution.  As I do,
I'll take a look at the allocation scheme in RecordingMemoryManager and see
if there is something reasonable I can do there.

In the meantime, is there any way that you can mark these tests as XFAIL in the
sanitizer case?

Thanks,
Andy

From: Alexey Samsonov [mailto:samsonov at google.com]
Sent: Tuesday, January 29, 2013 3:36 AM
To: LLVM Developers Mailing List
Cc: Kaylor, Andrew
Subject: Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

Hi!

I'm trying to run LLVM test suite under AddressSanitizer and get test
failures in:
    LLVM :: ExecutionEngine/MCJIT/simpletest-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-data-align-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-fp-no-external-funcs-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-global-init-nonzero-remote.ll

All of them fail with assertion:
lli:
/usr/local/google/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp:230:
void llvm::RuntimeDyldELF::resolveX86_64Relocation(const llvm::SectionEntry
&, uint64_t, uint64_t, uint32_t, int64_t): Assertion `RealOffset <=
(2147483647) && RealOffset >= (-2147483647-1)' failed.

The reason is that AddressSanitizer replaces system malloc with its own
allocator, which
allocates memory at "unusual" parts of heap and the difference between
pointers can be significant
(and doesn't fit in 32 bytes).

I add debug output to calculation of RealOffset in resolveX86_64Relocation:

    uint64_t  FinalAddress = Section.LoadAddress + Offset;
    int64_t RealOffset = *Placeholder + Value + Addend - FinalAddress;
    fprintf(stderr, "%x + %lx + %lx - %lx = %lx\n",
            *Placeholder, Value, Addend, FinalAddress, RealOffset);
    assert(RealOffset <= INT32_MAX && RealOffset >= INT32_MIN);




This is what I get for system malloc:



0 + 7fec9867a000 + 0 - 7fec9867a040 = ffffffffffffffc0
This is what I get for ASan allocator (which results in assert failure):



0 + 600c0000a8a0 + 0 - 6018000090a0 = fffffff400001800

--
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130129/1a12fa32/attachment.html>

Alexey Samsonov

2013-Jan-30 11:59 UTC

head link

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

Hi Andrew,

Looks like RecordingMemoryManager in lli just calls malloc() and it would
be strange to make assumptions (or enforce) that the difference between two
returned pointers in 64-bit
virtual address space will be fit into 32 bits. Can we do smth similar to
what Adhemerval proposed (see the special case in processRelocationRef for
ELF::R_PPC64_REL24 relocations)?


On Tue, Jan 29, 2013 at 9:40 PM, Kaylor, Andrew <andrew.kaylor at
intel.com>wrote:
>  Hi Alexey,****
>
> ** **
>
> I think the most likely way to resolve this is to have the
> RecordingMemoryManager do something more complex to manage its allocations
> in such a way as to guarantee that they are all within proximity of one
> another.  The code that is asserting is handling a relocation where code
> was generated to use a 32-bit relative offset in 64-bit code.  If the two
> sections involved really are more than a 32-bit offset apart then the
> generated code will not work.****
>
> ** **
>
> Alternatively, we could have MCJIT use whatever code generation option
> will prevent 32-bit relative relocations from being generated in the first
> place.  That would probably be preferable, but I haven’t had success trying
> to do that in limited efforts up to now.****
>
> ** **
>
> As it happens, I’m working with the ‘-use-remote’ option for lli this week
> trying to add support for actual out-of-process execution.  As I do, I’ll
> take a look at the allocation scheme in RecordingMemoryManager and see if
> there is something reasonable I can do there.****
>
> ** **
>
> In the meantime, is there any way that you can mark these tests as XFAIL
> in the sanitizer case?****
>
> ** **
>
> Thanks,****
>
> Andy****
>
> ** **
>
> *From:* Alexey Samsonov [mailto:samsonov at google.com]
> *Sent:* Tuesday, January 29, 2013 3:36 AM
> *To:* LLVM Developers Mailing List
> *Cc:* Kaylor, Andrew
> *Subject:* Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests****
>
> ** **
>
> Hi!****
>
> ** **
>
> I'm trying to run LLVM test suite under AddressSanitizer and get test
> failures in:****
>
>     LLVM :: ExecutionEngine/MCJIT/simpletest-remote.ll****
>
>     LLVM :: ExecutionEngine/MCJIT/test-data-align-remote.ll****
>
>     LLVM :: ExecutionEngine/MCJIT/test-fp-no-external-funcs-remote.ll****
>
>     LLVM :: ExecutionEngine/MCJIT/test-global-init-nonzero-remote.ll****
>
> ** **
>
> All of them fail with assertion:****
>
> lli:
>
/usr/local/google/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp:230:
> void llvm::RuntimeDyldELF::resolveX86_64Relocation(const llvm::SectionEntry
> &, uint64_t, uint64_t, uint32_t, int64_t): Assertion `RealOffset
<> (2147483647) && RealOffset >= (-2147483647-1)'
failed.****
>
> ** **
>
> The reason is that AddressSanitizer replaces system malloc with its own
> allocator, which****
>
> allocates memory at "unusual" parts of heap and the difference
between
> pointers can be significant****
>
> (and doesn't fit in 32 bytes).****
>
> ** **
>
> I add debug output to calculation of RealOffset in resolveX86_64Relocation:
> ****
>
> ** **
>
>     uint64_t  FinalAddress = Section.LoadAddress + Offset;****
>
>     int64_t RealOffset = *Placeholder + Value + Addend - FinalAddress;****
>
>     fprintf(stderr, "%x + %lx + %lx - %lx = %lx\n",****
>
>             *Placeholder, Value, Addend, FinalAddress, RealOffset);****
>
>     assert(RealOffset <= INT32_MAX && RealOffset >=
INT32_MIN);****
>
> ** **
>
> ** **
>
> This is what I get for system malloc:****
>
> ** **
>
> 0 + 7fec9867a000 + 0 - 7fec9867a040 = ffffffffffffffc0****
>
> This is what I get for ASan allocator (which results in assert failure):**
> **
>
> ** **
>
> 0 + 600c0000a8a0 + 0 - 6018000090a0 = fffffff400001800****
>
>   ** **
>
> --****
>
> Alexey Samsonov, MSK****
>


-- 
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130130/52c79680/attachment.html>

Possibly Parallel Threads

Search for more maybe matching threads

llvm dev - Jan 2013 - [LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

Possibly Parallel Threads