llvm dev - Nov 2009 - [LLVMdev] fastcc and ExecutionEngine::getPointerToFunction()

Yesterday I realised that if a Function has the calling convention set 
to "fast", then it is a bad idea to call it through the function
pointer
one gets from ExecutionEngine::getPointerToFunction().

The problem is that when calling it from my C++ program, the call will 
be made C-style, while the function expects to be called the fastcc way.

Could LLVM either warn the user of this, or solve it for the user?

One approach would be that getPointerToFunction() asserts that the 
function has default calling convention. This way, the user would be 
alerted.

Another approach would be that if getPointerToFunction() detects that 
the function does not have default calling convention, it produces a 
wrapper function, and returns a pointer to that.

My question is, is there code that would get into trouble because of
this?

For example, someone who is aware of this might be doing inline asm to 
produce a fastcc call through the pointer.

Or, someone might rely on the function actually being located on the 
returned address, and not necessarily calling it. This would break if 
the address to the wrapper is returned instead.

Any thoughts on this?

On a side note: wouldn't it be nice if the VerifierPass checks that 
calling conventions on calls and functions in a module match up?


Hans

Hans Wennborg wrote:
> Yesterday I realised that if a Function has the calling convention set
> to "fast", then it is a bad idea to call it through the function
pointer
> one gets from ExecutionEngine::getPointerToFunction().
>
> The problem is that when calling it from my C++ program, the call will
> be made C-style, while the function expects to be called the fastcc way.
>
> Could LLVM either warn the user of this, or solve it for the user?
>
> One approach would be that getPointerToFunction() asserts that the
> function has default calling convention. This way, the user would be
> alerted.
>
> Another approach would be that if getPointerToFunction() detects that
> the function does not have default calling convention, it produces a
> wrapper function, and returns a pointer to that.
>
> My question is, is there code that would get into trouble because of
> this?
Sure. You could be passing it in as a function pointer to another 
function, that's planning to call it with fastcc calling convention.

Or worse, you might be planning to call it with _stdcall (say, on 
Windows) and discarding the calling convention would be actively harmful.

In other words, please don't do this. Produce an llvm::Function to thunk 
if you must.

Nick
> For example, someone who is aware of this might be doing inline asm to
> produce a fastcc call through the pointer.
>
> Or, someone might rely on the function actually being located on the
> returned address, and not necessarily calling it. This would break if
> the address to the wrapper is returned instead.
>
> Any thoughts on this?
>
> On a side note: wouldn't it be nice if the VerifierPass checks that
> calling conventions on calls and functions in a module match up?
>
>
> Hans
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

On Nov 19, 2009, at 10:20 PM, Nick Lewycky wrote:
> 
> Sure. You could be passing it in as a function pointer to another 
> function, that's planning to call it with fastcc calling convention.
> 
> Or worse, you might be planning to call it with _stdcall (say, on 
> Windows) and discarding the calling convention would be actively harmful.
> 
> In other words, please don't do this. Produce an llvm::Function to
thunk
> if you must.
I very much agree with Nick here.  There are a multitude of things you may be
wanting to do and ways that you might want to call your fastcc function and
while this is, unfortunately, irritating and hard to track down the first couple
of times you make the mistake - the benefit is well worth it.

-eric