llvm dev - Feb 2010 - [LLVMdev] jit X86 target compilation callback bug

Hi!

We are running llvm jit x86 on MS Visual Studio 2005. It seems there
is a bug in asm code in function X86CompilationCallback in file
X86JITInfo.cpp. Current code sets stack pointer to invalid value in
instruction "and   esp,  16". Depending on current stack pointer value
it sometimes overwrites ecx and edx registers with next three lines.
We have fixed this problem by changing this instruction to "sub  esp,
8" (8 because this function needs 2 temp 32bit variables). Are we
correct?

Thanks and let us know.

Hello
> We are running llvm jit x86 on MS Visual Studio 2005.
Which version of llvm you're using?

-- 
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University

Hello
> We are running llvm jit x86 on MS Visual Studio 2005. It seems there
> is a bug in asm code in function X86CompilationCallback in file
> X86JITInfo.cpp. Current code sets stack pointer to invalid value in
> instruction "and   esp,  16". Depending on current stack pointer
value
> it sometimes overwrites ecx and edx registers with next three lines.
How so? The stack grows downwards, thus this realignment just equivalent
to "sub esp, some_value" with some_value being less than 16.
We don't use register save area inside the compilation callback,
however, since we're generating stubs "by hands".
> We have fixed this problem by changing this instruction to "sub  esp,
> 8" (8 because this function needs 2 temp 32bit variables).
What is "this" function?

-- 
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University

Hello again.

I still think that you are wrong. Realignement with and esp,-16 not
always changes stack poiner. If esp is already aligned to 16 byte
boundary, it will not change! Take a look at following example.
Assume esp has value 0x000001000 at start of X86CompilationCallback
function. Then execution of it will yield following esp values:

0x000000FFC - after push ebp
0x000000FFC - after mov ebp, esp
0x000000FF8 - after push eax
0x000000FF4 - after push edx // edx is pushed to address 0x000000FF4
0x000000FF0 - after push ecx // ecx is pushed to address 0x000000FF0
0x000000FF0 - after and esp, -16 // not changed!!!!

now next three instructions will corrupt pushed ecx and edx registers:

mov   eax, dword ptr [ebp+4]
mov   dword ptr [esp+4], eax
mov   dword ptr [esp], ebp

because [esp+4] - this is where pushed edx resides (0x000000FF0+4 = 0x000000FF4)
and [esp] is where pushed ecx resides (0x000000FF0)

instead of "and esp,-16" there would be "sub esp,8" then in
stack
there would be 8 bytes free to use (with [esp] and [esp+4]).
>
> What is "this" function?
>
By this I meant X86CompilationCallback function.

We are using 2.6 version. But same problem is in trunk also.


Kristaps.

On Tue, Feb 2, 2010 at 7:12 PM, Anton Korobeynikov
<anton at korobeynikov.info> wrote:
> Hello
>
>> We are running llvm jit x86 on MS Visual Studio 2005. It seems there
>> is a bug in asm code in function X86CompilationCallback in file
>> X86JITInfo.cpp. Current code sets stack pointer to invalid value in
>> instruction "and   esp,  16". Depending on current stack
pointer value
>> it sometimes overwrites ecx and edx registers with next three lines.
> How so? The stack grows downwards, thus this realignment just equivalent
> to "sub esp, some_value" with some_value being less than 16.
> We don't use register save area inside the compilation callback,
> however, since we're generating stubs "by hands".
>
>> We have fixed this problem by changing this instruction to "sub
 esp,
>> 8" (8 because this function needs 2 temp 32bit variables).
> What is "this" function?
>
> --
> With best regards, Anton Korobeynikov
> Faculty of Mathematics and Mechanics, Saint Petersburg State University
>