llvm dev - Mar 2012 - [LLVMdev] Access Violation using ExecutionEngine on 64-bit Windows 8 Consumer Preview

Hi everyone!

I've faced a strange problem after updating to Windows 8 Consumer
Preview recently. It seems that LLVM inserts 4 calls to the same
function at the start of generated code. The function's disassembly
(taken from nearby computer with Windows 7) is:

00000000773A0DD0  sub         rsp,10h
00000000773A0DD4  mov         qword ptr [rsp],r10
00000000773A0DD8  mov         qword ptr [rsp+8],r11
00000000773A0DDD  xor         r11,r11
00000000773A0DE0  lea         r10,[rsp+18h]
00000000773A0DE5  sub         r10,rax
00000000773A0DE8  cmovb       r10,r11
00000000773A0DEC  mov         r11,qword ptr gs:[10h]
00000000773A0DF5  cmp         r10,r11
00000000773A0DF8  jae         00000000773A0E10
00000000773A0DFA  and         r10w,0F000h
00000000773A0E00  lea         r11,[r11-1000h]
00000000773A0E07  mov         byte ptr [r11],0
00000000773A0E0B  cmp         r10,r11
00000000773A0E0E  jne         00000000773A0E00
00000000773A0E10  mov         r10,qword ptr [rsp]
00000000773A0E14  mov         r11,qword ptr [rsp+8]
00000000773A0E19  add         rsp,10h
00000000773A0E1D  ret

That function is called 3 or 4 times from my function like this:

0000000000C700A5  push        rax
0000000000C700A6  mov         esi,ecx
0000000000C700A8  sub         rsp,20h
0000000000C700AC  mov         rax,76CBC490h
0000000000C700B6  call        rax                                 ;
this is my call to DebugBreak() which goes first
0000000000C700B8  add         rsp,20h
0000000000C700BC  mov         eax,10h
0000000000C700C1  call        00000000773A0DD0
0000000000C700C6  sub         rsp,rax
0000000000C700C9  mov         r8,rsp
0000000000C700CC  mov         dword ptr [r8],0
0000000000C700D3  mov         eax,10h
0000000000C700D8  call        00000000773A0DD0
0000000000C700DD  sub         rsp,rax
0000000000C700E0  mov         rdx,rsp
0000000000C700E3  mov         dword ptr [rdx],0
0000000000C700E9  mov         eax,10h
0000000000C700EE  call        00000000773A0DD0
0000000000C700F3  sub         rsp,rax
0000000000C700F6  mov         rcx,rsp
0000000000C700F9  mov         dword ptr [rcx],0
0000000000C700FF  mov         eax,10h
0000000000C70104  call        00000000773A0DD0  ; 4 calls to the above function
0000000000C70109  sub         rsp,rax
0000000000C7010C  mov         dword ptr [rsp],0  ; here goes the
remaining code of my function
0000000000C70113  mov         dword ptr [r8],1
0000000000C7011A  mov         dword ptr [rdx],2
....


The problem is that in Windows 8 CP 4 calls to the first function
actually lead to nowhere. E.g. to the address in memory, that is not
allocated or improperly protected (either NX bit is set, or Read is
not set). Where should I start from to debug this behavior?

Best regards,
Victor Milovanov

Viktor, could you try my patch? I guess they are __chkstk.

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120220/137577.html

...Takumi

Takumi,
> Viktor, could you try my patch? I guess they are __chkstk.
4 are definitely too much :) If they are emitted - this is a bug.

-- 
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University

Hi, Takumi!

I tried your patch, and it did not help. Moreover, I tried to compile
under Windows 7 and copy files to Windows 8, and received the same
exception. So the problem seems to be in Windows 8 itself or some
non-portable code inside LLVM.

Could anyone tell me what LLVM code in ExecutionEngine is responsible
for allocating and protecting memory for generated native functions?

Best regards,
Victor Milovanov.

2012/3/2 NAKAMURA Takumi <geek4civic at
gmail.com>:
> Viktor, could you try my patch? I guess they are __chkstk.
>
>
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120220/137577.html
>
> ...Takumi