llvm dev - Oct 2015 - Add a mapping to a C++ lambda

Apologies for the noop question in advance (just getting started with
LLVM), and I'm not entirely sure if this is the right list to post to. is
it?

I have some lambda functions as member variables that I want to have my
LLVM language make calls to. I've added a mapping to them, but this
doesn't
seem to enable LLVM to resolve the functions. I asked on stackoverflow but
the suggestion there didn't help. The minimal test case to reproduce what
I'm trying to do is:

#include "llvm/ExecutionEngine/GenericValue.h"#include
"llvm/ExecutionEngine/Interpreter.h"#include
"llvm/IR/Constants.h"#include "llvm/IR/IRBuilder.h"#include
"llvm/Support/ManagedStatic.h"#include
"llvm/Support/TargetSelect.h"
using namespace llvm;
int main() {

  InitializeNativeTarget();

  LLVMContext Context;

  std::unique_ptr<Module> Owner =
make_unique<Module>("SomeModule", Context);
  Module *M = Owner.get();

  FunctionType *lambdaFT = FunctionType::get(Type::getInt32Ty(Context), false);
  Function *lambdaFN = Function::Create(lambdaFT,
Function::ExternalLinkage, "lambda", Owner.get());
  auto lambdaBody = []() { return 100; };

  Function *mainF =
cast<Function>(M->getOrInsertFunction("main",
Type::getInt32Ty(Context), (Type *) 0));

  BasicBlock *BB = BasicBlock::Create(Context, "EntryBlock", mainF);
  IRBuilder<> builder(BB);

  CallInst *lambdaRes = builder.CreateCall(lambdaFN, std::vector<Value
*>(), "lambdaRetVar");
  builder.CreateRet(lambdaRes);

  ExecutionEngine *EE = EngineBuilder(std::move(Owner)).create();
  EE->addGlobalMapping(lambdaFN, &lambdaBody);

  outs() << "We just constructed this LLVM module:\n\n" <<
*M;
  outs() << "\n\nRunning main: ";

  std::vector<GenericValue> noargs;
  GenericValue gv = EE->runFunction(mainF, noargs);

  outs() << "Result: " << gv.IntVal <<
"\n";
  llvm_shutdown();
  delete EE;
  return 0;}

The output is:

We just constructed this LLVM module:
; ModuleID = 'SomeModule'

declare i32 @lambda()

define i32 @main() {EntryBlock:
  %lambdaRetVar = call i32 @lambda()
  ret i32 %lambdaRetVar}
Running main:
LLVM ERROR: Tried to execute an unknown external function: lambda

The following suggestion was made on StackOveflow:
"
Your lambda body is a class. You must pass the address of its function call
operator, which you can do by converting it to a function pointer: auto
lambdaBody = +[]() { return 100; }; and passing it as a void*:
EE->addGlobalMapping(lambdaFN,
reinterpret_cast<void*>(lambdaBody));.
"
However, after doing that, I get the same LLVM error message. How do I get
LLVM to call to a lambda? I am likely to have some C functions loaded from
a dynamic lib (I'll know the name and signatures), is it the same process
to call these as it is for lambdas?

I know the Kaleidoscope example does std::sin so when it says it finds it
because it is within the same address space I thought this would work too...

No suggestions or code samples welcome.

Thanks in advance.
-- 
Courtney Robinson
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151027/6678eba6/attachment.html>

Courtney Robinson via llvm-dev <llvm-dev at lists.llvm.org>
wrote:
> How do I get LLVM to call to a lambda?
Depends on how that lambda is implemented -- I don't know how C++11
lambdas are implemented offhand. As a rule of thumb, if you want to
call a function from your LLVM program, you need to mark it `extern
"C"` -- this makes sure that the symbol in loaded ELF is literally
what you typed out (C-linkage, no C++ name mangling). In Kaleidoscope,
they are calling the C sin and cos functions.

If you have lingering doubts, load up the executable image in lldb and
(lldb) im loo -s lambda

to determine if that symbol is defined or not.

Hi Courtney,

There are at least two distinct issues here:

(1) The error you're seeing from LLVM ("Tried to execute an unknown
external function...") indicates that the mapping isn't working
correctly,
but your code for adding the global mapping looks good. There were some
issues with the global mapping on older versions of LLVM, and that may be
what you're running in to. What LLVM version and OS are you using?

(2) As Ramkumar noted, the lambda has two distinct addresses associated
with it: The closure (which I expect you get from &lambdaBody, though
I'd
have to check with a C++ expert), and the function (which I think of as
"&<lambda>::operator()", but I don't know if that's
really how C++ treats
it). You may be able to get the lambda to execute by setting up the mapping
to point to &<lambda>::operator() if you can figure out how to express
it
in C++, then passing the pointer to the closure as the sole argument to the
function (you'll need to change lambdaFN's signature from int() to
int(void*)). Whether this is guaranteed to work though I'm not sure.

Alternatively, if possible you can always wrap the lambda in a normal C
function to make it easy to access:

#include <...>

auto MyLamba = []() { return 100; }

int MyLambdaCaller() {
  return MyLambda();
}

Now you can include all of your code from before, but map 'lambdaFN' to
&MyLambdaCaller.

Hope this helps!

- Lang.

On Tue, Oct 27, 2015 at 1:21 AM, Courtney Robinson via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Apologies for the noop question in advance (just getting started with
> LLVM), and I'm not entirely sure if this is the right list to post to.
is
> it?
>
> I have some lambda functions as member variables that I want to have my
> LLVM language make calls to. I've added a mapping to them, but this
doesn't
> seem to enable LLVM to resolve the functions. I asked on stackoverflow but
> the suggestion there didn't help. The minimal test case to reproduce
what
> I'm trying to do is:
>
> #include "llvm/ExecutionEngine/GenericValue.h"#include
"llvm/ExecutionEngine/Interpreter.h"#include
"llvm/IR/Constants.h"#include "llvm/IR/IRBuilder.h"#include
"llvm/Support/ManagedStatic.h"#include
"llvm/Support/TargetSelect.h"
> using namespace llvm;
> int main() {
>
>   InitializeNativeTarget();
>
>   LLVMContext Context;
>
>   std::unique_ptr<Module> Owner =
make_unique<Module>("SomeModule", Context);
>   Module *M = Owner.get();
>
>   FunctionType *lambdaFT = FunctionType::get(Type::getInt32Ty(Context),
false);
>   Function *lambdaFN = Function::Create(lambdaFT,
Function::ExternalLinkage, "lambda", Owner.get());
>   auto lambdaBody = []() { return 100; };
>
>   Function *mainF =
cast<Function>(M->getOrInsertFunction("main",
Type::getInt32Ty(Context), (Type *) 0));
>
>   BasicBlock *BB = BasicBlock::Create(Context, "EntryBlock",
mainF);
>   IRBuilder<> builder(BB);
>
>   CallInst *lambdaRes = builder.CreateCall(lambdaFN, std::vector<Value
*>(), "lambdaRetVar");
>   builder.CreateRet(lambdaRes);
>
>   ExecutionEngine *EE = EngineBuilder(std::move(Owner)).create();
>   EE->addGlobalMapping(lambdaFN, &lambdaBody);
>
>   outs() << "We just constructed this LLVM module:\n\n"
<< *M;
>   outs() << "\n\nRunning main: ";
>
>   std::vector<GenericValue> noargs;
>   GenericValue gv = EE->runFunction(mainF, noargs);
>
>   outs() << "Result: " << gv.IntVal <<
"\n";
>   llvm_shutdown();
>   delete EE;
>   return 0;}
>
> The output is:
>
> We just constructed this LLVM module:
> ; ModuleID = 'SomeModule'
>
> declare i32 @lambda()
>
> define i32 @main() {EntryBlock:
>   %lambdaRetVar = call i32 @lambda()
>   ret i32 %lambdaRetVar}
> Running main:
> LLVM ERROR: Tried to execute an unknown external function: lambda
>
> The following suggestion was made on StackOveflow:
> "
> Your lambda body is a class. You must pass the address of its function
> call operator, which you can do by converting it to a function pointer:
auto
> lambdaBody = +[]() { return 100; }; and passing it as a void*:
EE->addGlobalMapping(lambdaFN,
> reinterpret_cast<void*>(lambdaBody));.
> "
> However, after doing that, I get the same LLVM error message. How do I get
> LLVM to call to a lambda? I am likely to have some C functions loaded from
> a dynamic lib (I'll know the name and signatures), is it the same
process
> to call these as it is for lambdas?
>
> I know the Kaleidoscope example does std::sin so when it says it finds it
> because it is within the same address space I thought this would work
too...
>
> No suggestions or code samples welcome.
>
> Thanks in advance.
> --
> Courtney Robinson
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151028/b9dfa135/attachment-0001.html>

On Wed, Oct 28, 2015 at 9:23 PM, Lang Hames <lhames at gmail.com> wrote:
> Hi Courtney,
>
> There are at least two distinct issues here:
>
> (1) The error you're seeing from LLVM ("Tried to execute an
unknown
> external function...") indicates that the mapping isn't working
correctly,
> but your code for adding the global mapping looks good. There were some
> issues with the global mapping on older versions of LLVM, and that may be
> what you're running in to. What LLVM version and OS are you using?
>
> (2) As Ramkumar noted, the lambda has two distinct addresses associated
> with it: The closure (which I expect you get from &lambdaBody, though
I'd
> have to check with a C++ expert), and the function (which I think of as
> "&<lambda>::operator()", but I don't know if
that's really how C++ treats
> it). You may be able to get the lambda to execute by setting up the mapping
> to point to &<lambda>::operator() if you can figure out how to
express it
> in C++, then passing the pointer to the closure as the sole argument to the
> function (you'll need to change lambdaFN's signature from int() to
> int(void*)). Whether this is guaranteed to work though I'm not sure.
>
Yep, pretty much everything there. Of course if you know the lambda is
stateless, you can convert it to a real function pointer (the common idiom
is to use unary '+' when you need to explicitly coerce a stateless
lambda
into a function pointer):

  addGlobalMapping(lambdaFN, +lambdaBody);

& there's no guaranteed way to get a usable free function for a stateful
lambda. As you mentioned, you could try/cheat by taking the address of the
operator() member function, reinterpret casting that to a non-member
function with the extra first parameter then passing that when you want to
call it.

>
> Alternatively, if possible you can always wrap the lambda in a normal C
> function to make it easy to access:
>
> #include <...>
>
> auto MyLamba = []() { return 100; }
>
> int MyLambdaCaller() {
>   return MyLambda();
> }
>
> Now you can include all of your code from before, but map
'lambdaFN' to
> &MyLambdaCaller.
>
> Hope this helps!
>
> - Lang.
>
> On Tue, Oct 27, 2015 at 1:21 AM, Courtney Robinson via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Apologies for the noop question in advance (just getting started with
>> LLVM), and I'm not entirely sure if this is the right list to post
to. is
>> it?
>>
>> I have some lambda functions as member variables that I want to have my
>> LLVM language make calls to. I've added a mapping to them, but this
doesn't
>> seem to enable LLVM to resolve the functions. I asked on stackoverflow
but
>> the suggestion there didn't help. The minimal test case to
reproduce what
>> I'm trying to do is:
>>
>> #include "llvm/ExecutionEngine/GenericValue.h"#include
"llvm/ExecutionEngine/Interpreter.h"#include
"llvm/IR/Constants.h"#include "llvm/IR/IRBuilder.h"#include
"llvm/Support/ManagedStatic.h"#include
"llvm/Support/TargetSelect.h"
>> using namespace llvm;
>> int main() {
>>
>>   InitializeNativeTarget();
>>
>>   LLVMContext Context;
>>
>>   std::unique_ptr<Module> Owner =
make_unique<Module>("SomeModule", Context);
>>   Module *M = Owner.get();
>>
>>   FunctionType *lambdaFT = FunctionType::get(Type::getInt32Ty(Context),
false);
>>   Function *lambdaFN = Function::Create(lambdaFT,
Function::ExternalLinkage, "lambda", Owner.get());
>>   auto lambdaBody = []() { return 100; };
>>
>>   Function *mainF =
cast<Function>(M->getOrInsertFunction("main",
Type::getInt32Ty(Context), (Type *) 0));
>>
>>   BasicBlock *BB = BasicBlock::Create(Context, "EntryBlock",
mainF);
>>   IRBuilder<> builder(BB);
>>
>>   CallInst *lambdaRes = builder.CreateCall(lambdaFN,
std::vector<Value *>(), "lambdaRetVar");
>>   builder.CreateRet(lambdaRes);
>>
>>   ExecutionEngine *EE = EngineBuilder(std::move(Owner)).create();
>>   EE->addGlobalMapping(lambdaFN, &lambdaBody);
>>
>>   outs() << "We just constructed this LLVM module:\n\n"
<< *M;
>>   outs() << "\n\nRunning main: ";
>>
>>   std::vector<GenericValue> noargs;
>>   GenericValue gv = EE->runFunction(mainF, noargs);
>>
>>   outs() << "Result: " << gv.IntVal <<
"\n";
>>   llvm_shutdown();
>>   delete EE;
>>   return 0;}
>>
>> The output is:
>>
>> We just constructed this LLVM module:
>> ; ModuleID = 'SomeModule'
>>
>> declare i32 @lambda()
>>
>> define i32 @main() {EntryBlock:
>>   %lambdaRetVar = call i32 @lambda()
>>   ret i32 %lambdaRetVar}
>> Running main:
>> LLVM ERROR: Tried to execute an unknown external function: lambda
>>
>> The following suggestion was made on StackOveflow:
>> "
>> Your lambda body is a class. You must pass the address of its function
>> call operator, which you can do by converting it to a function pointer:
auto
>> lambdaBody = +[]() { return 100; }; and passing it as a void*:
EE->addGlobalMapping(lambdaFN,
>> reinterpret_cast<void*>(lambdaBody));.
>> "
>> However, after doing that, I get the same LLVM error message. How do I
>> get LLVM to call to a lambda? I am likely to have some C functions
loaded
>> from a dynamic lib (I'll know the name and signatures), is it the
same
>> process to call these as it is for lambdas?
>>
>> I know the Kaleidoscope example does std::sin so when it says it finds
it
>> because it is within the same address space I thought this would work
too...
>>
>> No suggestions or code samples welcome.
>>
>> Thanks in advance.
>> --
>> Courtney Robinson
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151028/f0eb0f17/attachment.html>