edA-qa mort-ora-y
2011-Apr-11 15:50 UTC
[LLVMdev] appropriate for run-time compilation of DSL?
I wish to know whether LLVM is an appropriate choice for a project I'm working on (excuse me if this is the wrong list for this question). I have a domain-specific-language (DSL) that is currently compiled to a custom byte-code and run in a VM. This is written in C++. The DSL can call high-level functions which are actually callbacks into the current non-DSL environment. That is, these functions /leave/ the VM and work in the the normal code and return results back into the VM loop. For performance reasons I would now like to compile this code to native machine code. What I need is to be able to produce a binary block of data which can be distributed to other machines and run directly on them. That is, compilation would be on a central machine and worker machines would receive the code and execute it. From the docs and tutorial I'm unsure how I would go about doing this step. Can somebody confirm that LLVM can be used for what I want and point me somewhat in the right direction? -- edA-qa mort-ora-y -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Sign: Please digitally sign your emails. Encrypt: I'm also happy to receive encrypted mail. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20110411/cb4e59e8/attachment.sig>
Reid Kleckner
2011-Apr-11 16:24 UTC
[LLVMdev] appropriate for run-time compilation of DSL?
On Mon, Apr 11, 2011 at 11:50 AM, edA-qa mort-ora-y <eda-qa at disemia.com> wrote:> I wish to know whether LLVM is an appropriate choice for a project I'm > working on (excuse me if this is the wrong list for this question). > > I have a domain-specific-language (DSL) that is currently compiled to a > custom byte-code and run in a VM. This is written in C++. The DSL can > call high-level functions which are actually callbacks into the current > non-DSL environment. That is, these functions /leave/ the VM and work in > the the normal code and return results back into the VM loop.That should work perfectly fine. The LLVM JIT has support for declaring and calling out to native functions in the application through libffi. One catch is that if you want to call into C++ you will need to deal with a bit of the C++ ABI for any platforms you use. For example, at the LLVM IR level calls will use names mangled according to the ABI rules. You should be able to use clang as a library to help here. If you have straight C wrappers for your runtime functions, this complexity goes away.> For performance reasons I would now like to compile this code to native > machine code. What I need is to be able to produce a binary block of > data which can be distributed to other machines and run directly on > them. That is, compilation would be on a central machine and worker > machines would receive the code and execute it. From the docs and > tutorial I'm unsure how I would go about doing this step.There isn't great support for this in the LLVM JIT, but IIRC some clients have managed this by subclassing the JITMemoryManager to copy the code when the function has finished being emitted. I don't know if it's possible to ensure that the code will be position independent. Reid
edA-qa mort-ora-y
2011-Apr-12 03:37 UTC
[LLVMdev] appropriate for run-time compilation of DSL?
On 04/11/2011 06:24 PM, Reid Kleckner wrote:> That should work perfectly fine. The LLVM JIT has support for > declaring and calling out to native functions in the application > through libffi.How about support for a custom memory model? I know this sounds odd, but basically the variables need to map to a specific block of memory: the global heap. The enclosing program uses memcpy to push/pull values in/out of that memory. Would this be easy to do? Keep in mind I already generate byte-code, so if needed I could simply emit offsets into the IR for all the places I think it should be using.> with a bit of the C++ ABI for any platforms you use. For example, at > the LLVM IR level calls will use names mangled according to the ABII can use C-Wrappers if it is easier.> There isn't great support for this in the LLVM JIT, but IIRC some > clients have managed this by subclassing the JITMemoryManager to copy > the code when the function has finished being emitted. I don't know > if it's possible to ensure that the code will be position independent.I'll start my investigation here then, as this is actually quite critical to us -- we can't afford the final JIT/compilation overhead on our target machines. Note that I do presume that LLVM is extremely fast, but our requirements sit at the microsecond level, and we will have thousands of these things to compile in a day. Thank you for your assistance. -- edA-qa mort-ora-y -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Sign: Please digitally sign your emails. Encrypt: I'm also happy to receive encrypted mail. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20110412/83b0b1f2/attachment.sig>
Reasonably Related Threads
- [LLVMdev] appropriate for run-time compilation of DSL?
- [LLVMdev] appropriate for run-time compilation of DSL?
- [LLVMdev] appropriate for run-time compilation of DSL?
- [LLVMdev] UNREACHABLE executed at MCJIT.cpp:322!
- [LLVMdev] UNREACHABLE executed at MCJIT.cpp:322!