llvm dev - Aug 2011 - [LLVMdev] Using sret AND thiscall calling convention

Hi all,

I am trying to generate LLVM IR that calls an external C++ function
returning a structure by copy:
vec3 vec3::Cross(const vec3& iV) const;

Here is the LLVM IR that I am generating for win32 ABI, which says that the
first parameter should be a pointer to the return value:

%vec3 = type <{ float, float, float }>

define void @CPP_Return_Struct(%vec3* %v1, %vec3* %v2) inlinehint
alignstack(16) {
ENTRY:
  %"@0" = alloca %vec3, align 4
  call x86_thiscallcc void @vec3_Cross(%vec3* sret %"@0", %vec3* %v1,
%vec3*
%v2)
  ret void
}

declare x86_thiscallcc void @vec3_Cross(%vec3* sret, %vec3*, %vec3*) align 2

It seems that the code generated by the JIT takes the first parameter for
the this value because of the X86_thiscall calling convention. However, in
this particular case, the first parameter is the return value which is
tagged with the sret attribute.

I have manually modified the X86GenCallingConv.inc to treat the second
parameter as this value when the first has been tagged with sret attribute,
and it is working fine.

But I am not sure if it is a (known?) bug or if I do not generate the
correct LLVM IR.

I found nothing about this on the mailing list, I apologize if I missed any
important information.

Damien





-- 
View this message in context:
http://old.nabble.com/Using-sret-AND-thiscall-calling-convention-tp32242156p32242156.html
Sent from the LLVM - Dev mailing list archive at Nabble.com.

On Fri, Aug 12, 2011 at 1:11 AM, Damien Gleizes <gleizesd at gmail.com>
wrote:
>
> Hi all,
>
> I am trying to generate LLVM IR that calls an external C++ function
> returning a structure by copy:
> vec3 vec3::Cross(const vec3& iV) const;
>
> Here is the LLVM IR that I am generating for win32 ABI, which says that the
> first parameter should be a pointer to the return value:
>
> %vec3 = type <{ float, float, float }>
>
> define void @CPP_Return_Struct(%vec3* %v1, %vec3* %v2) inlinehint
> alignstack(16) {
> ENTRY:
>  %"@0" = alloca %vec3, align 4
>  call x86_thiscallcc void @vec3_Cross(%vec3* sret %"@0", %vec3*
%v1, %vec3*
> %v2)
>  ret void
> }
>
> declare x86_thiscallcc void @vec3_Cross(%vec3* sret, %vec3*, %vec3*) align
2
>
> It seems that the code generated by the JIT takes the first parameter for
> the this value because of the X86_thiscall calling convention. However, in
> this particular case, the first parameter is the return value which is
> tagged with the sret attribute.
>
> I have manually modified the X86GenCallingConv.inc to treat the second
> parameter as this value when the first has been tagged with sret attribute,
> and it is working fine.
>
> But I am not sure if it is a (known?) bug or if I do not generate the
> correct LLVM IR.
>
> I found nothing about this on the mailing list, I apologize if I missed any
> important information.
I would suggest avoiding any complicated calls from LLVM-generated
code into Visual Studio-generated code.  For example,
http://llvm.org/bugs/show_bug.cgi?id=5058 ,
http://llvm.org/bugs/show_bug.cgi?id=5064 .  Not sure about the
particular issue you're running into; looks like a bug at first
glance.  The file that mostly controls this sort of thing is
lib/Target/X86/X86CallingConv.td .

-Eli