llvm dev - Jan 2017 - Caller and Callee with different calling conventions

Hello,

If a call instruction and the function being called differ in their call
convention, InstructionCombine will insert a "store true, undef", then
CFGSimplify transforms this store to a llvm.trap() call, and finally,
during execution an InvalidInstruction exception is signaled.

Is there any reason not to detect the calling convention mismatch in the
verifier ? I would like to add such check.


Cheers,

Juan

http://llvm.org/docs/FAQ.html#why-does-instcombine-simplifycfg-turn-a-call-to-a-function-with-a-mismatched-calling-convention-into-unreachable-why-not-make-the-verifier-reject-it

Basically, you can imagine code like this:

enum CC { cdecl, stdcall, fastcall };
void callit(CC cc, void *fp) {
  switch (cc) {
  case cdecl: ((void(__cdecl*)())fp)(); break;
  case stdcall: ((void(__stdcall*)())fp)(); break;
  case fastcall: ((void(__fastcall*)())fp)(); break;
  }
}

If we inline 'callit', and discover the true target of fp, those calls
become direct, and the verifier should not reject them just because we
discovered additional information.

On Fri, Jan 6, 2017 at 1:32 AM, Juan Manuel Martinez Caamaño via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> If a call instruction and the function being called differ in their call
> convention, InstructionCombine will insert a "store true, undef",
then
> CFGSimplify transforms this store to a llvm.trap() call, and finally,
> during execution an InvalidInstruction exception is signaled.
>
> Is there any reason not to detect the calling convention mismatch in the
> verifier ? I would like to add such check.
>
>
> Cheers,
>
> Juan
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170106/cd3c2752/attachment.html>