llvm dev - Sep 2013 - [LLVMdev] VmKit - Error in j3/precompiler after a big surgery to MMTk part

Hi,

I replaced the MMTk code with a C-version MMTk code (I have been working 
on such AOT compiler for a while). The compiler should deal with most of 
the MMTk code base correctly - at least I get marksweep running with a 
tiny client). So I am trying to put the C version MMTk back with VmKit. 
I tried not to change much existing code except where necessary, such as 
name mangling and MMTkObject layout.

I am having the following error during building VmKit:

    [vmkit ./tools/trainer]: Pre-compiling bootstrap code
    Thread 0x110000000 received a SIGSEGV: either the VM code or an external
    native method is bogus. Aborting...
    Aborted
    make[1]: ***
    [/home/yilin/vmkit/vmkit/tools/trainer/Release+Asserts/Precompiled.bc]
    Error 134
    [vmkit .]: abort with error in ./tools/trainer
    make: *** [all] Error 1

The stacktrace from gdb is:

    Program received signal SIGSEGV, Segmentation fault.
    [Switching to Thread 0x1100ff700 (LWP 21670)]
    0x000000000107b180 in llvm::BumpPtrAllocator::Allocate(unsigned
    long, unsigned long) ()
    (gdb) bt
    #0  0x000000000107b180 in llvm::BumpPtrAllocator::Allocate(unsigned
    long, unsigned long) ()
    #1  0x00000000005ae35d in
    j3::JavaMethod::marshalArguments(vmkit::ThreadAllocator&,
    __va_list_tag*) ()
    #2  0x00000000005b04e4 in j3::JavaObject*
    j3::JavaMethod::invokeStaticAP<j3::JavaObject*, j3::JavaObject*
    (*)(j3::JavaConstantPool*, void*, void*)>(j3::Jnjvm*, j3::Class*,
    __va_list_tag*) ()
    #3  0x00000000005af054 in
    j3::JavaMethod::invokeJavaObjectStatic(j3::Jnjvm*, j3::Class*, ...) ()
    #4  0x00000000005cb662 in j3::Jnjvm::asciizToStr(char const*) ()
    #5  0x00000000005cbde4 in j3::Jnjvm::CreateArithmeticException() ()
    #6  0x00007ffff7f6316d in ?? ()
    #7  0x0000000000000000 in ?? ()

The main thread was waiting in "vmkit::VirtualMachine::waitForExit() 
()", so the application (HelloWorld) should have been started.

1. To me, the stack trace seems not showing where the original error 
happened. It seems an arithmetic error happened somewhere, VM was trying 
to throw an exception, but met another error. Is this correct? And how 
can I track where that actual error happened?

2. What does the precompiler do? I dont quite understand why it is 
needed here.

Any information would be very helpful. Thanks very much.

Regards,
Yi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130912/7ef23762/attachment.html>

Hi Yi,

The precompiler is in charge of translating the most common classes of
the Java runtime library to llvm code (typically, java.lang.Object),
in order to natively compile these classes and decrease the bootstrap
time. So, if you have a segmentation fault at this stage, it means
that you have broken something in the interface between MMTk and
VMKit. If it can helps you:

* during a first stage, we compile VMKit and J3 with a fake MMTk (we
replace gcmalloc by a simple malloc)
* we use the generated VMKit/J3 to natively compile MMTk
* we link this code with VMKit and replace malloc by the gcmalloc of
MMTk. At this stage, you have VMKit that use MMTk. In your case, the
compilation is correct up to this stage, meaning that VMKit is still
able to compile MMTk.
* we use VMKit/J3 (linked with MMTk) to precompile the rt.jar. So,
your compilation process breaks at this stage with a segmentation
fault caused by an arithmetic exception. As you have a segmentation
fault, I think that your C code of MMTk probably corrupt the memory,
either because you have a bug in your code, or because something goes
wrong in your modifications in the glue between VMKit and MMTk
(probably this second case).

As you have modified the object layout (in MMTkObject.h?), it probably
means that you use a different object layout? What kind of
modifications have you introduced? Because if you have changed the
object layout of Java objects, it can potentially impact all the code
of J3 and VMKit...

Gaël



2013/9/12 Yi Lin <qinsoon at gmail.com>:
> Hi,
>
> I replaced the MMTk code with a C-version MMTk code (I have been working on
> such AOT compiler for a while). The compiler should deal with most of the
> MMTk code base correctly - at least I get marksweep running with a tiny
> client). So I am trying to put the C version MMTk back with VmKit. I tried
> not to change much existing code except where necessary, such as name
> mangling and MMTkObject layout.
>
> I am having the following error during building VmKit:
>
> [vmkit ./tools/trainer]: Pre-compiling bootstrap code
> Thread 0x110000000 received a SIGSEGV: either the VM code or an external
> native method is bogus. Aborting...
> Aborted
> make[1]: ***
> [/home/yilin/vmkit/vmkit/tools/trainer/Release+Asserts/Precompiled.bc]
Error
> 134
> [vmkit .]: abort with error in ./tools/trainer
> make: *** [all] Error 1
>
> The stacktrace from gdb is:
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x1100ff700 (LWP 21670)]
> 0x000000000107b180 in llvm::BumpPtrAllocator::Allocate(unsigned long,
> unsigned long) ()
> (gdb) bt
> #0  0x000000000107b180 in llvm::BumpPtrAllocator::Allocate(unsigned long,
> unsigned long) ()
> #1  0x00000000005ae35d in
> j3::JavaMethod::marshalArguments(vmkit::ThreadAllocator&,
__va_list_tag*) ()
> #2  0x00000000005b04e4 in j3::JavaObject*
> j3::JavaMethod::invokeStaticAP<j3::JavaObject*, j3::JavaObject*
> (*)(j3::JavaConstantPool*, void*, void*)>(j3::Jnjvm*, j3::Class*,
> __va_list_tag*) ()
> #3  0x00000000005af054 in
j3::JavaMethod::invokeJavaObjectStatic(j3::Jnjvm*,
> j3::Class*, ...) ()
> #4  0x00000000005cb662 in j3::Jnjvm::asciizToStr(char const*) ()
> #5  0x00000000005cbde4 in j3::Jnjvm::CreateArithmeticException() ()
> #6  0x00007ffff7f6316d in ?? ()
> #7  0x0000000000000000 in ?? ()
>
> The main thread was waiting in "vmkit::VirtualMachine::waitForExit()
()", so
> the application (HelloWorld) should have been started.
>
> 1. To me, the stack trace seems not showing where the original error
> happened. It seems an arithmetic error happened somewhere, VM was trying to
> throw an exception, but met another error. Is this correct? And how can I
> track where that actual error happened?
>
> 2. What does the precompiler do? I dont quite understand why it is needed
> here.
>
> Any information would be very helpful. Thanks very much.
>
> Regards,
> Yi
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>


-- 
-------------------------------------------------------------------
Gaël Thomas, Associate Professor, UPMC
http://pagesperso-systeme.lip6.fr/Gael.Thomas/
-------------------------------------------------------------------

Hi Gaël,

Since I am replacing Java MMTk with C MMTk, accordingly I also changed 
the build process.

On 12/09/13 19:53 , Gaël Thomas wrote:
> Hi Yi,
>
> The precompiler is in charge of translating the most common classes of
> the Java runtime library to llvm code (typically, java.lang.Object),
> in order to natively compile these classes and decrease the bootstrap
> time. So, if you have a segmentation fault at this stage, it means
> that you have broken something in the interface between MMTk and
> VMKit. If it can helps you:
>
> * during a first stage, we compile VMKit and J3 with a fake MMTk (we
> replace gcmalloc by a simple malloc)
> * we use the generated VMKit/J3 to natively compile MMTk
> * we link this code with VMKit and replace malloc by the gcmalloc of
> MMTk. At this stage, you have VMKit that use MMTk. In your case, the
> compilation is correct up to this stage, meaning that VMKit is still
> able to compile MMTk.
The above stages seem no longer needed. I also removed the fake MMTk 
methods.
The error happened in precompiling, so I assume the compilation of C 
MMTk and linking with J3 appears to be fine (though there might be
problems)
> * we use VMKit/J3 (linked with MMTk) to precompile the rt.jar. So,
> your compilation process breaks at this stage with a segmentation
> fault caused by an arithmetic exception. As you have a segmentation
> fault, I think that your C code of MMTk probably corrupt the memory,
> either because you have a bug in your code, or because something goes
> wrong in your modifications in the glue between VMKit and MMTk
> (probably this second case).
>
> As you have modified the object layout (in MMTkObject.h?), it probably
> means that you use a different object layout? What kind of
> modifications have you introduced? Because if you have changed the
> object layout of Java objects, it can potentially impact all the code
> of J3 and VMKit...
Since MMTk is no longer generated by j3, but by my AOT compiler, the 
object layout is different. I didn't modify MMTkObject.h, but I added my 
own header to allow glue code correctly (hopefully) access fields and 
construct 'objects' for MMTk. I noticed that vmkit/j3 holds reference to
MMTk objects, but my guess was they wouldn't access MMTk objects without 
going through the glue code. So I only modified the glue code to adapt 
to new layout, not the other parts. But still, the 'objects' in C MMTk 
are not Java objects, and do not follow j3 Java object layout. I am not 
sure if j3 implies that MMTk objects are its own java object. Is this 
problematic? If so, I need to reconsider my solution.

And thanks for the reply, its very helpful.

Regards,
Yi
>
> Gaël
>
>
>
> 2013/9/12 Yi Lin <qinsoon at gmail.com>:
>> Hi,
>>
>> I replaced the MMTk code with a C-version MMTk code (I have been
working on
>> such AOT compiler for a while). The compiler should deal with most of
the
>> MMTk code base correctly - at least I get marksweep running with a tiny
>> client). So I am trying to put the C version MMTk back with VmKit. I
tried
>> not to change much existing code except where necessary, such as name
>> mangling and MMTkObject layout.
>>
>> I am having the following error during building VmKit:
>>
>> [vmkit ./tools/trainer]: Pre-compiling bootstrap code
>> Thread 0x110000000 received a SIGSEGV: either the VM code or an
external
>> native method is bogus. Aborting...
>> Aborted
>> make[1]: ***
>> [/home/yilin/vmkit/vmkit/tools/trainer/Release+Asserts/Precompiled.bc]
Error
>> 134
>> [vmkit .]: abort with error in ./tools/trainer
>> make: *** [all] Error 1
>>
>> The stacktrace from gdb is:
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 0x1100ff700 (LWP 21670)]
>> 0x000000000107b180 in llvm::BumpPtrAllocator::Allocate(unsigned long,
>> unsigned long) ()
>> (gdb) bt
>> #0  0x000000000107b180 in llvm::BumpPtrAllocator::Allocate(unsigned
long,
>> unsigned long) ()
>> #1  0x00000000005ae35d in
>> j3::JavaMethod::marshalArguments(vmkit::ThreadAllocator&,
__va_list_tag*) ()
>> #2  0x00000000005b04e4 in j3::JavaObject*
>> j3::JavaMethod::invokeStaticAP<j3::JavaObject*, j3::JavaObject*
>> (*)(j3::JavaConstantPool*, void*, void*)>(j3::Jnjvm*, j3::Class*,
>> __va_list_tag*) ()
>> #3  0x00000000005af054 in
j3::JavaMethod::invokeJavaObjectStatic(j3::Jnjvm*,
>> j3::Class*, ...) ()
>> #4  0x00000000005cb662 in j3::Jnjvm::asciizToStr(char const*) ()
>> #5  0x00000000005cbde4 in j3::Jnjvm::CreateArithmeticException() ()
>> #6  0x00007ffff7f6316d in ?? ()
>> #7  0x0000000000000000 in ?? ()
>>
>> The main thread was waiting in
"vmkit::VirtualMachine::waitForExit() ()", so
>> the application (HelloWorld) should have been started.
>>
>> 1. To me, the stack trace seems not showing where the original error
>> happened. It seems an arithmetic error happened somewhere, VM was
trying to
>> throw an exception, but met another error. Is this correct? And how can
I
>> track where that actual error happened?
>>
>> 2. What does the precompiler do? I dont quite understand why it is
needed
>> here.
>>
>> Any information would be very helpful. Thanks very much.
>>
>> Regards,
>> Yi
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
>