Anders Alexandersson
2004-May-11 06:26 UTC
[LLVMdev] Follow-up on: Dynamic updates of current executed code
Hello!
I am coming back to the below discussion again, regarding the LLVM support of
Ruby dynamics. The initial problem description is as follow, to refresh your
memory :) After that my questions come:
-----------BEGIN Initial problem description---------------
Problem is though, that the Ruby compiler is integrated in the compilation of
the program being executed, to be able to parse & compile dynamic code at
run-time. Therefore the calls to ExecutionEngine::getPointerToGlobal(F) need to
be made in LLVM code. Here is a detailed simplistic example in pseudocode of
what we want to do:
First Ruby code is entered at run-time, received as a string, parsed and
compiled into the following code:
%--------------
; External function
declare int %printf(sbyte*, ...)
; Custom function
int %puts_kernel( sbyte* %string )
{
%tmp.0 = call int (sbyte*, ...)* %printf( sbyte* %string )
ret int 0
}
%-------------
This code is represented in the string variable
%dynamically_compiled_function_code below:
%----------
%dynamically_compiled_function_code = internal constant [LENGTH x sbyte]
c"--String with the function code--\0A\00"
; Table of function pointer(s)
%kernel = type { int ( sbyte* )* }
int %main() {
; Create the kernel in memory, and get pointer to first function pointer
%theKernel = malloc %kernel
%FirstFunctionPTR = getelementptr %kernel* %theKernel, long 0, ubyte 0
;Load code
%myNewFunction = %getPointerToGlobal(%dynamically_compiled_function_code)
; Write memory address of myNewFunction() into kernel struct
store RETURNTYPE (PARAMTYPE*)* %myNewFunction, RETURNTYPE (PARAMTYPE*)**
%FirstFunctionPTR
;Any code using first function element in %kernel is now_
;using dynamically updated function!?
ret int 0
}
%-----------
The questionmark is at this pseudocode row:
%myNewFunction = %getPointerToGlobal(%dynamically_compiled_function_code)
Is there an llvm version of the getPointerToGlobal() function as outlined, and
can the %myNewFunction pointer be used as described?
Also, does the getPointerToGlobal() take human readable code (.ll) or only
binary byte code (.bc)? Is there a specification of how to write binary byte
code directly, so we do not have to externally call the llvm-as utility?
Best regards
Anders
-----------END Initial problem description---------------
-------------------Previous Discussion:
>> Is there an llvm version of the getPointerToGlobal() function as
>> outlined, and can the %myNewFunction pointer be used as described?
>Yes, see llvm/lib/ExecutionEngine/ExecutionEngine.cpp . It will compile
>the given Function* to machine code, and return a pointer to you. In
>your LLVM code, you will have to first cast your function pointer before
>storing it, but it should work as you have it.
1) Can a modified version of ParseAssemblyFile(InputFilename) that takes a
pointer to LLVM code in memory instead be used to generate a Function* for use
by the getPointerToGlobal()? I understand that it currently returns a Module*.
Is it possible to make it return an F* instead, and if so how would the
implementation roughly look like?
2) Can this function be compiled into LLVM independently and used as a
stand-alone function by the run-time compiler?
>One thing is that you need to have a pointer to the currently-running
>instance of ExecutionEngine to call methods on it... a easy quick hack
>would be to create a static variable that the ExecutionEngine
>constructor writes its own pointer (this) into, and have a C function
>(in ExecutionEngine.cpp) that returns that pointer. Your code can then
>access the currently-running ExecutionEngine through that pointer. There
>should be a cleaner solution to this...
We aim for an integrated binary solution with compiler-application-LLVMjitter,
(if at all possible), so my question is:
1) How is it possible to make LLVM calls to C++ methods? Can the Ruby-LLVM
compiler, applied on itself thus residing in LLVM-form, call the
getPointerToGlobal() using the pointer to the currently-running instance of
ExecutionEngine from the
fast hack
above? :) In this process it is passing the F* from the modified
ParseAssemblyFile(InputFilename), finally getting a pointer to the function,
which can be called using standard LLVM syntax?
(&)
>Here's an idea:
>In tools/llvm-as/llvm-as.cpp, you can see how the assembly parser really
>does its job: it calls ParseAssemblyFile(InputFilename) which returns it
>a Module*.
>ParseAssemblyFile is defined in llvm/lib/AsmParser/Parser.cpp; something
>similar to it, but one that accepts a string instead of a filename could
>be implemented.
>However, there is a catch (as I see it, Chris may correct me):
>The entry to the parser is to parse a whole Module. The new function
>that you write may reference other functions. When you parse a Module,
>it has to be complete, so functions that are not defined in the module
>are "external". Having a definition for those functions will make
those
>external functions different (in terms of their location in memory) and
>hence different value for Function*, which will mess with the
>ExecutionEngine as it uses a map Function* => memory address of
>translation. Other problems would be with types, same idea.
>(Is this making sense?)
>A solution would be perhaps to use a different entry point into a parser
>that would just parse a Function instead of a whole Module. It should
>then resolve the types it finds to the already-parsed Module in memory.
>I don't think this has been done yet.
>--
>Misha Brukman :: http://misha.brukman.net :: http://llvm.cs.uiuc.edu
I hope you are able to roughly describe this scenario! We do not aim for
implementing it at this point, but only to know if it is possible, and roughly
how.
Best regards
Anders Alexandersson
RubyComp project
----------------------------------------------------------------
Anders Alexandersson
Masters student at the special year of Software Engineering, HTU Trollhättan
E-mail: anders.alexandersson at student.htu.se
Chris Lattner
2004-May-11 10:06 UTC
[LLVMdev] Follow-up on: Dynamic updates of current executed code
On Tue, 11 May 2004, Anders Alexandersson wrote:> The questionmark is at this pseudocode row: > %myNewFunction = %getPointerToGlobal(%dynamically_compiled_function_code) > > Is there an llvm version of the getPointerToGlobal() function as > outlined, and can the %myNewFunction pointer be used as described? > Also, does the getPointerToGlobal() take human readable code (.ll) or > only binary byte code (.bc)? Is there a specification of how to write > binary byte code directly, so we do not have to externally call the > llvm-as utility?getPointerToGlobal can not take either of those (a .bc or .ll file). It must take an LLVM "Function*" to tell it what to compile. I think the way you really want to approach this is to write your ruby compiler in C++, and have a public function that you expose "compileThisStringOfRubyCodeToAFunctionAndReturnAPointerToIt(const char*)". Your compiler would lex and parse the ruby code in the string, translate it to LLVM (to a Function *) and then ask the JIT to compile it for you, returning a pointer to the function. -Chris -- http://llvm.cs.uiuc.edu/ http://www.nondot.org/~sabre/Projects/