thr3ads.net - llvm dev - [LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen? [Oct 2013]

If this information is useful, please help other people find it:
Share via:

Yaron Keren

2013-Oct-23 09:50 UTC

[LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen?

If it's a Windows-only thing the correct tests would be:

  if (NumBytes >= 4096 && STI.isOSWindows()) {

and

  if (Subtarget->isTargetWindows())

where

  bool isOSWindows() const { return TargetTriple.isOSWindows(); }

Yaron



2013/10/23 Andrew MacPherson <andrew.macp at gmail.com>
> Glad that helped! As I understand it __chkstk is always required on
> Windows regardless of output type, I had meant to file a bug about this but
> had apparently forgotten to do so. I think the check needs to be that the
> target is Windows and ignore the output type, Linux and OSX don't use
this.
>
> Cheers,
> Andrew
>

On Wed, Oct 23, 2013 at 11:32 AM, Yaron Keren <yaron.keren at gmail.com>
wrote:
> YES, this is the problem!
>
> The program work ok, even a 5x larger version works well.
>
> Clearly the _chkstk calls must be emitted with ELF target on Windows as
> well - why not?
>
> I'd like to make a patch and fix this right.
>
> I experimented with both changes and practically only the
> lib/Target/X86/X86ISelLowering.cpp fixes the problem.  The other change
> lib/Target/X86/X86FrameLowering.cpp was not required to fix the problem
> thus it is probably required for other reasons.
>
> So, should I patch both tests?
> Is the correct patch removing the test isTargetCOFF() completely?
> Or enabling it for both COFF or ELF tarrgets?
> I mean - is there any X86 target that does NOT require this stack checking?
>
> Yaron
>
>
>
> 2013/10/23 Andrew MacPherson <andrew.macp at gmail.com>
>
>> Hi Yaron,
>>
>> If you're outputting ELF on Windows this sounds like an issue we
ran into
>> where __chkstk calls weren't being output in the assembly due to an
>> explicit check for COFF output. Once stack allocations in a given
function
>> exceeded some amount we'd get exactly this kind of crash in the
function
>> initialization.
>>
>> If you take a look for isTargetCOFF() in
>> lib/Target/X86/X86ISelLowering.cpp and
lib/Target/X86/X86FrameLowering.cpp
>> you should be able to remove that check to force __chkstk output to see
if
>> that helps.
>>
>> Cheers,
>> Andrew
>>
>
>
> On Wed, Oct 23, 2013 at 1:22 AM, Yaron Keren <yaron.keren at
gmail.com>wrote:
>
>> Yes, this is correct code address accessing bad data address.
>>
>> However, there is no other relocation before .text or near it. I'll
send
>> you the full debug printout, maybe you'll note something.
>>
>> The problem could be result of something else entirely else than the
>> linker such as some library initialization code that by chance worked
with
>> smaller code but fails now.
>>
>> I need to debug and see what's going on. The trouble is no debug
>> information. Maybe I can do without the source code information and
debug
>> the assembly but without any symbols it's really a challenge to
understand
>> anything. I did try to make MCJIT emit debug info but for some reason
>> attached gdb did not understand it. Maybe this could be solved.
>>
>> I assumed there may be some limitations around 31-32 bits as there are
>> various int32 members in the ELF structure, but that's far far
away.
>> Problems start at .text size of about 150K.
>>
>> Yaron
>>
>>
>>
>>
>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at intel.com>
>>
>>>  So it looks like 0x0A3600D1 is a good code address and there’s no
>>> problem executing the code there, but 0x00BC7680 is a bad data
address.  Is
>>> that correct?****
>>>
>>> ** **
>>>
>>> If so, this is almost certainly a relocation problem.  You just
need to
>>> find a relocation that writes an entry (probably a relative offset)
at
>>> 0x0A3600D1+the size of the instruction at that address.****
>>>
>>> ** **
>>>
>>> BTW, what I said before about not being aware of any size
limitations
>>> wasn’t quite correct.  If you have enough code and data that we end
up
>>> putting sections at addresses that are more than 2GB apart we’ll
have
>>> problems, but you should see an assertion in that case.  That can
happen if
>>> we weren’t able to get the address we requested from
allocateMappedMemory,
>>> but it doesn’t look like that’s what’s happening here.****
>>>
>>> ** **
>>>
>>> -Andy****
>>>
>>> ** **
>>>
>>> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>> *Sent:* Tuesday, October 22, 2013 1:41 PM
>>>
>>> *To:* Kaylor, Andrew
>>> *Cc:* <llvmdev at cs.uiuc.edu>
>>> *Subject:* Re: Size limitations in MCJIT / ELF Dynamic Linker/ ELF
>>> codegen?****
>>>
>>> ** **
>>>
>>> Hi,****
>>>
>>> ** **
>>>
>>> Thanks for your ideas.****
>>>
>>> ** **
>>>
>>> Memory allocation already exceeds 2x64K in the "working"
case so it's
>>> not the condition of allocating more than 64K. To be sure I had
modified
>>> SectionMemoryManager::allocateSection to allocate four time the
required
>>> memory but it did not trigger more crashes.I debugged through the
>>> allocation code including the Win32 code and it seems to work well.
I have
>>> also tried disabling the MemGroup.FreeMem cache which did not
matter.***
>>> *
>>>
>>> ** **
>>>
>>> An added assert for no Stubs to the end of
RuntimeDyldImpl::loadObject**
>>> **
>>>
>>>       processRelocationRef(SectionID, *i, *obj, LocalSections,
>>> LocalSymbols,****
>>>
>>>                 Stubs);****
>>>
>>>       assert(!Stubs.size());****
>>>
>>> indeed caught nothing = no stubs created.****
>>>
>>> ** **
>>>
>>> Disabling (de)registerEH did not help.****
>>>
>>> ** **
>>>
>>> Looking at relocations and sections printouts, the exception
is:****
>>>
>>> ** **
>>>
>>> Unhandled exception at 0x0A3600D1 :****
>>>
>>> 0xC0000005: Access violation writing location 0x00BC7680.****
>>>
>>> ** **
>>>
>>> which is right after the start of .text:****
>>>
>>> ** **
>>>
>>> emitSection SectionID: 1 Name: .text obj addr: 0A3F1350 new addr:
>>> 0A360000 DataSize: 253203 StubBufSize: 0 Allocate: 253203****
>>>
>>> ...****
>>>
>>> Resolving relocations Section #1        0A360000****
>>>
>>> ** **
>>>
>>> so at least it is running code but tries to write a wrong
location.****
>>>
>>> Another run exhibits similar crash, still in .text but somewhat
later.**
>>> **
>>>
>>> ** **
>>>
>>> I have checked and the function address I'm running is located
in .text
>>> towards the end, as expected since it's the last function added
to the
>>> Module.****
>>>
>>> ** **
>>>
>>> Also I speculated that if it crashes when .text crosses 128K but
no, it
>>> happens when it's larger.****
>>>
>>> ** **
>>>
>>> I had attached gdb to the process hoping it will show more
information
>>> but it showed even less information than the Visual C++
debugger.****
>>>
>>> ** **
>>>
>>> Out of ideas... ****
>>>
>>> ** **
>>>
>>> Yaron****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at intel.com>****
>>>
>>>  I would guess that it’s crashing somewhere in the generated code. 
On
>>> Windows we don’t have a way to get call stacks to the generated
code
>>> (though if you want to try it on Linux, that should work).  You can
>>> probably look at the address where the crash is occurring and
verify that
>>> it is in the generated code.****
>>>
>>>  ****
>>>
>>> There are a couple of things I would look for.****
>>>
>>>  ****
>>>
>>> First, I’d take a look at the SectionMemoryManager allocation
handling.
>>> The fact that the problem is code size dependent strongly points in
this
>>> direction.  It may be that SectionMemoryManager does something
wrong when
>>> it hits a page boundary or something.****
>>>
>>>  ****
>>>
>>> Second, I’d look at the relocation processing.  If it is generating
any
>>> stubs, that would be a potential problem spot, but it shouldn’t be
>>> generating any stubs.  So the obvious thing to look at is whether
any of
>>> the relocations are writing to the spot where the crash occurs.****
>>>
>>>  ****
>>>
>>> -Andy****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>> *Sent:* Tuesday, October 22, 2013 10:17 AM
>>> *To:* Kaylor, Andrew
>>> *Cc:* <llvmdev at cs.uiuc.edu>
>>> *Subject:* Re: Size limitations in MCJIT / ELF Dynamic Linker/ ELF
>>> codegen?****
>>>
>>>  ****
>>>
>>> OS is Windows 7 64 bit OS, compiler is 32 bit Visual C++ 2012 with
32
>>> bit.****
>>>
>>> The target which is i686-pc-mingw32-elf so I can use the ELF
dynamic
>>> loader. ****
>>>
>>> Code model, relocation model and and memory manager are whatever
default
>>> for this - did not modify.****
>>>
>>>  ****
>>>
>>> The Module comes from clang. The source is 1000 or more lines
repeating
>>> C++ code in one big function:****
>>>
>>>  ****
>>>
>>>   A+1;****
>>>
>>>   A*B.t();****
>>>
>>>  ****
>>>
>>> where A and B are matrices from Armadillo
http://arma.sourceforge.net/.
>>> This a stress and performance test due to the large number of EH
and
>>> temporary objects created.****
>>>
>>>  ****
>>>
>>> I am using the Engine Builder and MCJIT unmodified (except the
>>> multi-modules patches which are not relevant as there is only one
module)
>>> like this:****
>>>
>>>  ****
>>>
>>>   OwningPtr<llvm::ExecutionEngine>
EE(llvm::EngineBuilder(M)****
>>>
>>>                                          
.setErrorStr(&Error)****
>>>
>>>                                           .setUseMCJIT(true)****
>>>
>>>                                           .create());****
>>>
>>>  ****
>>>
>>> to run the function either ****
>>>
>>>  ****
>>>
>>>   llvm::Function *F = M->getFunction(Name);****
>>>
>>>   void *FN = EE->getPointerToFunction(F);****
>>>
>>> or****
>>>
>>>   uint64_t FN = EE->getFunctionAddress(Name);****
>>>
>>>  ****
>>>
>>> followed by ****
>>>
>>>  ****
>>>
>>>  ((void (*)())FN)();****
>>>
>>> or****
>>>
>>>   EE->runFunction(F,
std::vector<llvm::GenericValue>());****
>>>
>>>  ****
>>>
>>> all work the same with smaller about 1000 lines of the above code
module
>>> and crash the same with more code. The call stack is unhelpful
Visual C++
>>> says: Frames below may be incorrect and/or missing which indicates
a real
>>> problem with it. I have tried to provide less stack space (default
is 10M)
>>> for the compiled program without any change.****
>>>
>>>  ****
>>>
>>> Yaron****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at intel.com>****
>>>
>>>   I’m not aware of such a limitation.****
>>>
>>>  ****
>>>
>>> What architecture, code model and relocation model are you using? 
Are
>>> you using the SectionMemoryManager?****
>>>
>>>  ****
>>>
>>> -Andy****
>>>
>>>  ****
>>>
>>> *From**:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>> *Sent**:* Tuesday, October 22, 2013 8:12 AM
>>> *To**:* <llvmdev at cs.uiuc.edu>; Kaylor, Andrew
>>> *Subject**:* Size limitations in MCJIT / ELF Dynamic Linker/ ELF
codegen
>>> ?****
>>>
>>>  ****
>>>
>>> I'm running in MCJIT a module generated from one C++ function.
Every
>>> line of the source function uses C++ classes and may throw an
>>> exception. As long as there are less than (about) 1000 lines,
everything
>>> works. With more lines the compiled code crashes when running it,
with
>>> no sensible stack trace.****
>>>
>>>  ****
>>>
>>> Is there any kind of hard-coded size limitation in MCJIT / ELF
Dynamic
>>> Linker / ELF codegen / number of EH states in a function ? ****
>>>
>>>  ****
>>>
>>> I did browse the code but could not find anything obvious. ****
>>>
>>>  ****
>>>
>>> Yaron****
>>>
>>>  ****
>>>
>>>   ****
>>>
>>>  ** **
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131023/4df3bd4c/attachment.html>

Reid Kleckner

2013-Oct-23 16:37 UTC

head link

[LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen?

This is the right fix if Cygwin wants calls to __chkstk.  Otherwise you'll
want TargetTriple.isOSMSVCRT().


On Wed, Oct 23, 2013 at 2:50 AM, Yaron Keren <yaron.keren at gmail.com>
wrote:
> If it's a Windows-only thing the correct tests would be:
>
>   if (NumBytes >= 4096 && STI.isOSWindows()) {
>
> and
>
>   if (Subtarget->isTargetWindows())
>
> where
>
>    bool isOSWindows() const { return TargetTriple.isOSWindows(); }
>
> Yaron
>
>
>
> 2013/10/23 Andrew MacPherson <andrew.macp at gmail.com>
>
>> Glad that helped! As I understand it __chkstk is always required on
>> Windows regardless of output type, I had meant to file a bug about this
but
>> had apparently forgotten to do so. I think the check needs to be that
the
>> target is Windows and ignore the output type, Linux and OSX don't
use this.
>>
>> Cheers,
>> Andrew
>>
>
>
> On Wed, Oct 23, 2013 at 11:32 AM, Yaron Keren <yaron.keren at
gmail.com>wrote:
>
>> YES, this is the problem!
>>
>> The program work ok, even a 5x larger version works well.
>>
>> Clearly the _chkstk calls must be emitted with ELF target on Windows as
>> well - why not?
>>
>> I'd like to make a patch and fix this right.
>>
>> I experimented with both changes and practically only the
>> lib/Target/X86/X86ISelLowering.cpp fixes the problem.  The other change
>> lib/Target/X86/X86FrameLowering.cpp was not required to fix the problem
>> thus it is probably required for other reasons.
>>
>> So, should I patch both tests?
>> Is the correct patch removing the test isTargetCOFF() completely?
>> Or enabling it for both COFF or ELF tarrgets?
>> I mean - is there any X86 target that does NOT require this stack
>> checking?
>>
>> Yaron
>>
>>
>>
>> 2013/10/23 Andrew MacPherson <andrew.macp at gmail.com>
>>
>>> Hi Yaron,
>>>
>>> If you're outputting ELF on Windows this sounds like an issue
we ran
>>> into where __chkstk calls weren't being output in the assembly
due to an
>>> explicit check for COFF output. Once stack allocations in a given
function
>>> exceeded some amount we'd get exactly this kind of crash in the
function
>>> initialization.
>>>
>>> If you take a look for isTargetCOFF() in
>>> lib/Target/X86/X86ISelLowering.cpp and
lib/Target/X86/X86FrameLowering.cpp
>>> you should be able to remove that check to force __chkstk output to
see if
>>> that helps.
>>>
>>> Cheers,
>>> Andrew
>>>
>>
>>
>> On Wed, Oct 23, 2013 at 1:22 AM, Yaron Keren <yaron.keren at
gmail.com>wrote:
>>
>>> Yes, this is correct code address accessing bad data address.
>>>
>>> However, there is no other relocation before .text or near it.
I'll send
>>> you the full debug printout, maybe you'll note something.
>>>
>>> The problem could be result of something else entirely else than
the
>>> linker such as some library initialization code that by chance
worked with
>>> smaller code but fails now.
>>>
>>> I need to debug and see what's going on. The trouble is no
debug
>>> information. Maybe I can do without the source code information and
debug
>>> the assembly but without any symbols it's really a challenge to
understand
>>> anything. I did try to make MCJIT emit debug info but for some
reason
>>> attached gdb did not understand it. Maybe this could be solved.
>>>
>>> I assumed there may be some limitations around 31-32 bits as there
are
>>> various int32 members in the ELF structure, but that's far far
away.
>>> Problems start at .text size of about 150K.
>>>
>>> Yaron
>>>
>>>
>>>
>>>
>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at intel.com>
>>>
>>>>  So it looks like 0x0A3600D1 is a good code address and there’s
no
>>>> problem executing the code there, but 0x00BC7680 is a bad data
address.  Is
>>>> that correct?****
>>>>
>>>> ** **
>>>>
>>>> If so, this is almost certainly a relocation problem.  You just
need to
>>>> find a relocation that writes an entry (probably a relative
offset) at
>>>> 0x0A3600D1+the size of the instruction at that address.****
>>>>
>>>> ** **
>>>>
>>>> BTW, what I said before about not being aware of any size
limitations
>>>> wasn’t quite correct.  If you have enough code and data that we
end up
>>>> putting sections at addresses that are more than 2GB apart
we’ll have
>>>> problems, but you should see an assertion in that case.  That
can happen if
>>>> we weren’t able to get the address we requested from
allocateMappedMemory,
>>>> but it doesn’t look like that’s what’s happening here.****
>>>>
>>>> ** **
>>>>
>>>> -Andy****
>>>>
>>>> ** **
>>>>
>>>> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>>> *Sent:* Tuesday, October 22, 2013 1:41 PM
>>>>
>>>> *To:* Kaylor, Andrew
>>>> *Cc:* <llvmdev at cs.uiuc.edu>
>>>> *Subject:* Re: Size limitations in MCJIT / ELF Dynamic Linker/
ELF
>>>> codegen?****
>>>>
>>>> ** **
>>>>
>>>> Hi,****
>>>>
>>>> ** **
>>>>
>>>> Thanks for your ideas.****
>>>>
>>>> ** **
>>>>
>>>> Memory allocation already exceeds 2x64K in the
"working" case so it's
>>>> not the condition of allocating more than 64K. To be sure I had
modified
>>>> SectionMemoryManager::allocateSection to allocate four time the
required
>>>> memory but it did not trigger more crashes.I debugged through
the
>>>> allocation code including the Win32 code and it seems to work
well. I have
>>>> also tried disabling the MemGroup.FreeMem cache which did not
matter.**
>>>> **
>>>>
>>>> ** **
>>>>
>>>> An added assert for no Stubs to the end of
RuntimeDyldImpl::loadObject*
>>>> ***
>>>>
>>>>       processRelocationRef(SectionID, *i, *obj, LocalSections,
>>>> LocalSymbols,****
>>>>
>>>>                 Stubs);****
>>>>
>>>>       assert(!Stubs.size());****
>>>>
>>>> indeed caught nothing = no stubs created.****
>>>>
>>>> ** **
>>>>
>>>> Disabling (de)registerEH did not help.****
>>>>
>>>> ** **
>>>>
>>>> Looking at relocations and sections printouts, the exception
is:****
>>>>
>>>> ** **
>>>>
>>>> Unhandled exception at 0x0A3600D1 :****
>>>>
>>>> 0xC0000005: Access violation writing location 0x00BC7680.****
>>>>
>>>> ** **
>>>>
>>>> which is right after the start of .text:****
>>>>
>>>> ** **
>>>>
>>>> emitSection SectionID: 1 Name: .text obj addr: 0A3F1350 new
addr:
>>>> 0A360000 DataSize: 253203 StubBufSize: 0 Allocate: 253203****
>>>>
>>>> ...****
>>>>
>>>> Resolving relocations Section #1        0A360000****
>>>>
>>>> ** **
>>>>
>>>> so at least it is running code but tries to write a wrong
location.****
>>>>
>>>> Another run exhibits similar crash, still in .text but somewhat
later.*
>>>> ***
>>>>
>>>> ** **
>>>>
>>>> I have checked and the function address I'm running is
located in .text
>>>> towards the end, as expected since it's the last function
added to the
>>>> Module.****
>>>>
>>>> ** **
>>>>
>>>> Also I speculated that if it crashes when .text crosses 128K
but no, it
>>>> happens when it's larger.****
>>>>
>>>> ** **
>>>>
>>>> I had attached gdb to the process hoping it will show more
information
>>>> but it showed even less information than the Visual C++
debugger.****
>>>>
>>>> ** **
>>>>
>>>> Out of ideas... ****
>>>>
>>>> ** **
>>>>
>>>> Yaron****
>>>>
>>>> ** **
>>>>
>>>> ** **
>>>>
>>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at
intel.com>****
>>>>
>>>>  I would guess that it’s crashing somewhere in the generated
code.  On
>>>> Windows we don’t have a way to get call stacks to the generated
code
>>>> (though if you want to try it on Linux, that should work).  You
can
>>>> probably look at the address where the crash is occurring and
verify that
>>>> it is in the generated code.****
>>>>
>>>>  ****
>>>>
>>>> There are a couple of things I would look for.****
>>>>
>>>>  ****
>>>>
>>>> First, I’d take a look at the SectionMemoryManager allocation
>>>> handling.  The fact that the problem is code size dependent
strongly points
>>>> in this direction.  It may be that SectionMemoryManager does
something
>>>> wrong when it hits a page boundary or something.****
>>>>
>>>>  ****
>>>>
>>>> Second, I’d look at the relocation processing.  If it is
generating any
>>>> stubs, that would be a potential problem spot, but it shouldn’t
be
>>>> generating any stubs.  So the obvious thing to look at is
whether any of
>>>> the relocations are writing to the spot where the crash
occurs.****
>>>>
>>>>  ****
>>>>
>>>> -Andy****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>>> *Sent:* Tuesday, October 22, 2013 10:17 AM
>>>> *To:* Kaylor, Andrew
>>>> *Cc:* <llvmdev at cs.uiuc.edu>
>>>> *Subject:* Re: Size limitations in MCJIT / ELF Dynamic Linker/
ELF
>>>> codegen?****
>>>>
>>>>  ****
>>>>
>>>> OS is Windows 7 64 bit OS, compiler is 32 bit Visual C++ 2012
with 32
>>>> bit.****
>>>>
>>>> The target which is i686-pc-mingw32-elf so I can use the ELF
dynamic
>>>> loader. ****
>>>>
>>>> Code model, relocation model and and memory manager are
whatever
>>>> default for this - did not modify.****
>>>>
>>>>  ****
>>>>
>>>> The Module comes from clang. The source is 1000 or more lines
repeating
>>>> C++ code in one big function:****
>>>>
>>>>  ****
>>>>
>>>>   A+1;****
>>>>
>>>>   A*B.t();****
>>>>
>>>>  ****
>>>>
>>>> where A and B are matrices from Armadillo
http://arma.sourceforge.net/.
>>>> This a stress and performance test due to the large number of
EH and
>>>> temporary objects created.****
>>>>
>>>>  ****
>>>>
>>>> I am using the Engine Builder and MCJIT unmodified (except the
>>>> multi-modules patches which are not relevant as there is only
one module)
>>>> like this:****
>>>>
>>>>  ****
>>>>
>>>>   OwningPtr<llvm::ExecutionEngine>
EE(llvm::EngineBuilder(M)****
>>>>
>>>>                                          
.setErrorStr(&Error)****
>>>>
>>>>                                          
.setUseMCJIT(true)****
>>>>
>>>>                                           .create());****
>>>>
>>>>  ****
>>>>
>>>> to run the function either ****
>>>>
>>>>  ****
>>>>
>>>>   llvm::Function *F = M->getFunction(Name);****
>>>>
>>>>   void *FN = EE->getPointerToFunction(F);****
>>>>
>>>> or****
>>>>
>>>>   uint64_t FN = EE->getFunctionAddress(Name);****
>>>>
>>>>  ****
>>>>
>>>> followed by ****
>>>>
>>>>  ****
>>>>
>>>>  ((void (*)())FN)();****
>>>>
>>>> or****
>>>>
>>>>   EE->runFunction(F,
std::vector<llvm::GenericValue>());****
>>>>
>>>>  ****
>>>>
>>>> all work the same with smaller about 1000 lines of the above
code
>>>> module and crash the same with more code. The call stack is
unhelpful
>>>> Visual C++ says: Frames below may be incorrect and/or missing
which
>>>> indicates a real problem with it. I have tried to provide less
stack space
>>>> (default is 10M) for the compiled program without any
change.****
>>>>
>>>>  ****
>>>>
>>>> Yaron****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at
intel.com>****
>>>>
>>>>   I’m not aware of such a limitation.****
>>>>
>>>>  ****
>>>>
>>>> What architecture, code model and relocation model are you
using?  Are
>>>> you using the SectionMemoryManager?****
>>>>
>>>>  ****
>>>>
>>>> -Andy****
>>>>
>>>>  ****
>>>>
>>>> *From**:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>>> *Sent**:* Tuesday, October 22, 2013 8:12 AM
>>>> *To**:* <llvmdev at cs.uiuc.edu>; Kaylor, Andrew
>>>> *Subject**:* Size limitations in MCJIT / ELF Dynamic Linker/
ELF
>>>> codegen?****
>>>>
>>>>  ****
>>>>
>>>> I'm running in MCJIT a module generated from one C++
function. Every
>>>> line of the source function uses C++ classes and may throw an
>>>> exception. As long as there are less than (about) 1000 lines,
everything
>>>> works. With more lines the compiled code crashes when running
it, with
>>>> no sensible stack trace.****
>>>>
>>>>  ****
>>>>
>>>> Is there any kind of hard-coded size limitation in MCJIT / ELF
Dynamic
>>>> Linker / ELF codegen / number of EH states in a function ? ****
>>>>
>>>>  ****
>>>>
>>>> I did browse the code but could not find anything obvious. ****
>>>>
>>>>  ****
>>>>
>>>> Yaron****
>>>>
>>>>  ****
>>>>
>>>>   ****
>>>>
>>>>  ** **
>>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131023/6d92a653/attachment.html>

Yaron Keren

2013-Oct-23 17:12 UTC

head link

[LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen?

I have not much personal experience Cygwin with but the code inside the
above condition considers MingW/Cygwin identical:

    if (Is64Bit) {
      if (STI.isTargetCygMing())
        StackProbeSymbol = "___chkstk";
      else {
        StackProbeSymbol = "__chkstk";
        isSPUpdateNeeded = true;
      }
    } else if (STI.isTargetCygMing())
      StackProbeSymbol = "_alloca";
    else
      StackProbeSymbol = "_chkstk";

I'll make a patch.

Yaron



2013/10/23 Reid Kleckner <rnk at google.com>
> This is the right fix if Cygwin wants calls to __chkstk.  Otherwise
you'll
> want TargetTriple.isOSMSVCRT().


On Wed, Oct 23, 2013 at 2:50 AM, Yaron Keren <yaron.keren at gmail.com>
wrote:
> If it's a Windows-only thing the correct tests would be:
>
>   if (NumBytes >= 4096 && STI.isOSWindows()) {
>
> and
>
>   if (Subtarget->isTargetWindows())
>
> where
>
>    bool isOSWindows() const { return TargetTriple.isOSWindows(); }
>
> Yaron
>
>
>
> 2013/10/23 Andrew MacPherson <andrew.macp at gmail.com>
>
>> Glad that helped! As I understand it __chkstk is always required on
>> Windows regardless of output type, I had meant to file a bug about this
but
>> had apparently forgotten to do so. I think the check needs to be that
the
>> target is Windows and ignore the output type, Linux and OSX don't
use this.
>>
>> Cheers,
>> Andrew
>>
>
>
> On Wed, Oct 23, 2013 at 11:32 AM, Yaron Keren <yaron.keren at
gmail.com>wrote:
>
>> YES, this is the problem!
>>
>> The program work ok, even a 5x larger version works well.
>>
>> Clearly the _chkstk calls must be emitted with ELF target on Windows as
>> well - why not?
>>
>> I'd like to make a patch and fix this right.
>>
>> I experimented with both changes and practically only the
>> lib/Target/X86/X86ISelLowering.cpp fixes the problem.  The other change
>> lib/Target/X86/X86FrameLowering.cpp was not required to fix the problem
>> thus it is probably required for other reasons.
>>
>> So, should I patch both tests?
>> Is the correct patch removing the test isTargetCOFF() completely?
>> Or enabling it for both COFF or ELF tarrgets?
>> I mean - is there any X86 target that does NOT require this stack
>> checking?
>>
>> Yaron
>>
>>
>>
>> 2013/10/23 Andrew MacPherson <andrew.macp at gmail.com>
>>
>>> Hi Yaron,
>>>
>>> If you're outputting ELF on Windows this sounds like an issue
we ran
>>> into where __chkstk calls weren't being output in the assembly
due to an
>>> explicit check for COFF output. Once stack allocations in a given
function
>>> exceeded some amount we'd get exactly this kind of crash in the
function
>>> initialization.
>>>
>>> If you take a look for isTargetCOFF() in
>>> lib/Target/X86/X86ISelLowering.cpp and
lib/Target/X86/X86FrameLowering.cpp
>>> you should be able to remove that check to force __chkstk output to
see if
>>> that helps.
>>>
>>> Cheers,
>>> Andrew
>>>
>>
>>
>> On Wed, Oct 23, 2013 at 1:22 AM, Yaron Keren <yaron.keren at
gmail.com>wrote:
>>
>>> Yes, this is correct code address accessing bad data address.
>>>
>>> However, there is no other relocation before .text or near it.
I'll send
>>> you the full debug printout, maybe you'll note something.
>>>
>>> The problem could be result of something else entirely else than
the
>>> linker such as some library initialization code that by chance
worked with
>>> smaller code but fails now.
>>>
>>> I need to debug and see what's going on. The trouble is no
debug
>>> information. Maybe I can do without the source code information and
debug
>>> the assembly but without any symbols it's really a challenge to
understand
>>> anything. I did try to make MCJIT emit debug info but for some
reason
>>> attached gdb did not understand it. Maybe this could be solved.
>>>
>>> I assumed there may be some limitations around 31-32 bits as there
are
>>> various int32 members in the ELF structure, but that's far far
away.
>>> Problems start at .text size of about 150K.
>>>
>>> Yaron
>>>
>>>
>>>
>>>
>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at intel.com>
>>>
>>>>  So it looks like 0x0A3600D1 is a good code address and there’s
no
>>>> problem executing the code there, but 0x00BC7680 is a bad data
address.  Is
>>>> that correct?****
>>>>
>>>> ** **
>>>>
>>>> If so, this is almost certainly a relocation problem.  You just
need to
>>>> find a relocation that writes an entry (probably a relative
offset) at
>>>> 0x0A3600D1+the size of the instruction at that address.****
>>>>
>>>> ** **
>>>>
>>>> BTW, what I said before about not being aware of any size
limitations
>>>> wasn’t quite correct.  If you have enough code and data that we
end up
>>>> putting sections at addresses that are more than 2GB apart
we’ll have
>>>> problems, but you should see an assertion in that case.  That
can happen if
>>>> we weren’t able to get the address we requested from
allocateMappedMemory,
>>>> but it doesn’t look like that’s what’s happening here.****
>>>>
>>>> ** **
>>>>
>>>> -Andy****
>>>>
>>>> ** **
>>>>
>>>> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>>> *Sent:* Tuesday, October 22, 2013 1:41 PM
>>>>
>>>> *To:* Kaylor, Andrew
>>>> *Cc:* <llvmdev at cs.uiuc.edu>
>>>> *Subject:* Re: Size limitations in MCJIT / ELF Dynamic Linker/
ELF
>>>> codegen?****
>>>>
>>>> ** **
>>>>
>>>> Hi,****
>>>>
>>>> ** **
>>>>
>>>> Thanks for your ideas.****
>>>>
>>>> ** **
>>>>
>>>> Memory allocation already exceeds 2x64K in the
"working" case so it's
>>>> not the condition of allocating more than 64K. To be sure I had
modified
>>>> SectionMemoryManager::allocateSection to allocate four time the
required
>>>> memory but it did not trigger more crashes.I debugged through
the
>>>> allocation code including the Win32 code and it seems to work
well. I have
>>>> also tried disabling the MemGroup.FreeMem cache which did not
matter.**
>>>> **
>>>>
>>>> ** **
>>>>
>>>> An added assert for no Stubs to the end of
RuntimeDyldImpl::loadObject*
>>>> ***
>>>>
>>>>       processRelocationRef(SectionID, *i, *obj, LocalSections,
>>>> LocalSymbols,****
>>>>
>>>>                 Stubs);****
>>>>
>>>>       assert(!Stubs.size());****
>>>>
>>>> indeed caught nothing = no stubs created.****
>>>>
>>>> ** **
>>>>
>>>> Disabling (de)registerEH did not help.****
>>>>
>>>> ** **
>>>>
>>>> Looking at relocations and sections printouts, the exception
is:****
>>>>
>>>> ** **
>>>>
>>>> Unhandled exception at 0x0A3600D1 :****
>>>>
>>>> 0xC0000005: Access violation writing location 0x00BC7680.****
>>>>
>>>> ** **
>>>>
>>>> which is right after the start of .text:****
>>>>
>>>> ** **
>>>>
>>>> emitSection SectionID: 1 Name: .text obj addr: 0A3F1350 new
addr:
>>>> 0A360000 DataSize: 253203 StubBufSize: 0 Allocate: 253203****
>>>>
>>>> ...****
>>>>
>>>> Resolving relocations Section #1        0A360000****
>>>>
>>>> ** **
>>>>
>>>> so at least it is running code but tries to write a wrong
location.****
>>>>
>>>> Another run exhibits similar crash, still in .text but somewhat
later.*
>>>> ***
>>>>
>>>> ** **
>>>>
>>>> I have checked and the function address I'm running is
located in .text
>>>> towards the end, as expected since it's the last function
added to the
>>>> Module.****
>>>>
>>>> ** **
>>>>
>>>> Also I speculated that if it crashes when .text crosses 128K
but no, it
>>>> happens when it's larger.****
>>>>
>>>> ** **
>>>>
>>>> I had attached gdb to the process hoping it will show more
information
>>>> but it showed even less information than the Visual C++
debugger.****
>>>>
>>>> ** **
>>>>
>>>> Out of ideas... ****
>>>>
>>>> ** **
>>>>
>>>> Yaron****
>>>>
>>>> ** **
>>>>
>>>> ** **
>>>>
>>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at
intel.com>****
>>>>
>>>>  I would guess that it’s crashing somewhere in the generated
code.  On
>>>> Windows we don’t have a way to get call stacks to the generated
code
>>>> (though if you want to try it on Linux, that should work).  You
can
>>>> probably look at the address where the crash is occurring and
verify that
>>>> it is in the generated code.****
>>>>
>>>>  ****
>>>>
>>>> There are a couple of things I would look for.****
>>>>
>>>>  ****
>>>>
>>>> First, I’d take a look at the SectionMemoryManager allocation
>>>> handling.  The fact that the problem is code size dependent
strongly points
>>>> in this direction.  It may be that SectionMemoryManager does
something
>>>> wrong when it hits a page boundary or something.****
>>>>
>>>>  ****
>>>>
>>>> Second, I’d look at the relocation processing.  If it is
generating any
>>>> stubs, that would be a potential problem spot, but it shouldn’t
be
>>>> generating any stubs.  So the obvious thing to look at is
whether any of
>>>> the relocations are writing to the spot where the crash
occurs.****
>>>>
>>>>  ****
>>>>
>>>> -Andy****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>>> *Sent:* Tuesday, October 22, 2013 10:17 AM
>>>> *To:* Kaylor, Andrew
>>>> *Cc:* <llvmdev at cs.uiuc.edu>
>>>> *Subject:* Re: Size limitations in MCJIT / ELF Dynamic Linker/
ELF
>>>> codegen?****
>>>>
>>>>  ****
>>>>
>>>> OS is Windows 7 64 bit OS, compiler is 32 bit Visual C++ 2012
with 32
>>>> bit.****
>>>>
>>>> The target which is i686-pc-mingw32-elf so I can use the ELF
dynamic
>>>> loader. ****
>>>>
>>>> Code model, relocation model and and memory manager are
whatever
>>>> default for this - did not modify.****
>>>>
>>>>  ****
>>>>
>>>> The Module comes from clang. The source is 1000 or more lines
repeating
>>>> C++ code in one big function:****
>>>>
>>>>  ****
>>>>
>>>>   A+1;****
>>>>
>>>>   A*B.t();****
>>>>
>>>>  ****
>>>>
>>>> where A and B are matrices from Armadillo
http://arma.sourceforge.net/.
>>>> This a stress and performance test due to the large number of
EH and
>>>> temporary objects created.****
>>>>
>>>>  ****
>>>>
>>>> I am using the Engine Builder and MCJIT unmodified (except the
>>>> multi-modules patches which are not relevant as there is only
one module)
>>>> like this:****
>>>>
>>>>  ****
>>>>
>>>>   OwningPtr<llvm::ExecutionEngine>
EE(llvm::EngineBuilder(M)****
>>>>
>>>>                                          
.setErrorStr(&Error)****
>>>>
>>>>                                          
.setUseMCJIT(true)****
>>>>
>>>>                                           .create());****
>>>>
>>>>  ****
>>>>
>>>> to run the function either ****
>>>>
>>>>  ****
>>>>
>>>>   llvm::Function *F = M->getFunction(Name);****
>>>>
>>>>   void *FN = EE->getPointerToFunction(F);****
>>>>
>>>> or****
>>>>
>>>>   uint64_t FN = EE->getFunctionAddress(Name);****
>>>>
>>>>  ****
>>>>
>>>> followed by ****
>>>>
>>>>  ****
>>>>
>>>>  ((void (*)())FN)();****
>>>>
>>>> or****
>>>>
>>>>   EE->runFunction(F,
std::vector<llvm::GenericValue>());****
>>>>
>>>>  ****
>>>>
>>>> all work the same with smaller about 1000 lines of the above
code
>>>> module and crash the same with more code. The call stack is
unhelpful
>>>> Visual C++ says: Frames below may be incorrect and/or missing
which
>>>> indicates a real problem with it. I have tried to provide less
stack space
>>>> (default is 10M) for the compiled program without any
change.****
>>>>
>>>>  ****
>>>>
>>>> Yaron****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>> 2013/10/22 Kaylor, Andrew <andrew.kaylor at
intel.com>****
>>>>
>>>>   I’m not aware of such a limitation.****
>>>>
>>>>  ****
>>>>
>>>> What architecture, code model and relocation model are you
using?  Are
>>>> you using the SectionMemoryManager?****
>>>>
>>>>  ****
>>>>
>>>> -Andy****
>>>>
>>>>  ****
>>>>
>>>> *From**:* Yaron Keren [mailto:yaron.keren at gmail.com]
>>>> *Sent**:* Tuesday, October 22, 2013 8:12 AM
>>>> *To**:* <llvmdev at cs.uiuc.edu>; Kaylor, Andrew
>>>> *Subject**:* Size limitations in MCJIT / ELF Dynamic Linker/
ELF
>>>> codegen?****
>>>>
>>>>  ****
>>>>
>>>> I'm running in MCJIT a module generated from one C++
function. Every
>>>> line of the source function uses C++ classes and may throw an
>>>> exception. As long as there are less than (about) 1000 lines,
everything
>>>> works. With more lines the compiled code crashes when running
it, with
>>>> no sensible stack trace.****
>>>>
>>>>  ****
>>>>
>>>> Is there any kind of hard-coded size limitation in MCJIT / ELF
Dynamic
>>>> Linker / ELF codegen / number of EH states in a function ? ****
>>>>
>>>>  ****
>>>>
>>>> I did browse the code but could not find anything obvious. ****
>>>>
>>>>  ****
>>>>
>>>> Yaron****
>>>>
>>>>  ****
>>>>
>>>>   ****
>>>>
>>>>  ** **
>>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131023/40776cf5/attachment.html>

Possibly Parallel Threads

Search for more possibly parallel threads

llvm dev - Oct 2013 - [LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen?

[LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen?

[LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen?

[LLVMdev] Size limitations in MCJIT / ELF Dynamic Linker/ ELF codegen?

Possibly Parallel Threads