Hi again, and thanks (Reid) for your fast response: Yes, it works!!! Only changing the order of libraries in the Makefile. Nowaday I have my software with the capability of compile assembly, bytecode (from buffer and file) and link them with a set of libraries. It seems to work perfectly (I don't generate code yet). My real aim is to have a process (host) with execute several no-jit binaries (guest) each one in his own thread (not forked!! each one with a main function). Guest and host have interdependencies in both ways. This is a part of my doctoral degree. Before now, I was using TCC (Tiny C Compiler), but it have a big lack: it isn't reentrant. It can only have one generated program at time. For this reason I agree to use LLVM (two days ago) and rebuild all the project. 8-| LLVM Makefile systen is great!! But I have no time to change nowadays all my current Makefiles. Maybe later. I have noticed that LLVM have memory laks; exactly a poor main with a simple "return 0;" linked with the LLVM libraries report memory laks using valgrind. Is it normal? Now I have other problem: I have a Module and I need generate a iostream (memory) with native x86 code (maybe elf/coff) to be executed later (into the guest process space, without fork!!). I studied llc and lli, but they don't help me much. Any idea? Are there any guy working in some like that? A sample code will be greatfully :-) -----Mensaje original----- De: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] En nombre de Reid Spencer Enviado el: jueves, 30 de diciembre de 2004 21:47 Para: LLVM Developers Mailing List Asunto: Re: [LLVMdev] Primer with LLVM On Thu, 2004-12-30 at 11:14, Francisco Puentes wrote:> Hi, everybody: >Hi Francisco> > I am a beginner with LLVM, in fact today was the first day that I use it.Welcome!> > I have several questions about LLVM:If you haven't already, a good place to start is the Getting Started Guide, at http://llvm.cs.uiuc.edu/docs/GettingStarted.html> Can I use LLVM to compile several files (bytecode), scripts (char*) andlink> them with external libraries generating *only* one executable (all in > memory)?I'm not exactly sure what you're trying to do, but I believe you can do what you want. The llvm-link tool will link together multiple bytecode files. lli will generate code in memory (JIT compilation) and execute it. Since the bulk of the implementation of these tools is in the LLVM libraries, you can create your own process to do the same sorts of things. I'm not quite sure what you mean by "scripts (char*)" but if you mean LLVM assembly then this is possible to. The ParseAssemblyFile function (include/llvm/Assembly/Parser.h) can be used to turn an LLVM assembly into a Module* which is then suitable for linking via the linker interface (include/llvm/Linker.h).> > Can I invoke externals functions from a guest (LLVM generated) code which > exist in the host code (the code that execute the first one)?Yes. The lli interpreter/JIT compiler will resolve symbols within its own executable or dynamically via a loadable plug-in (-load option). Some platforms don't support this currently (Cygwin for example).> > And a problem: > > I made a little aplication with use the functions llvm::LinkModules and > llvm::LinkLibraries that, I suppouse, exist in libLLVMLinker.a archive,that is correct.> but > gcc linker reports 'undefined' errors for this functions.The only thing I can think of is dependencies that libLLVMLinker has. You will probably want a linke line that looks something like: gcc -o myapp myapp.o -lLLVMLinker -lLLVMArchive -lLLVMBCReader \ -lLLVMBCWriter -lLLVMCore -lLLVMSupport -lLLVMbzip2 -lLLVMSystem To get examples of these library specifications, look at the llvm-ld and gccld tools' Makefiles.> I don't use the > makefile system from LLVM because is so complex to incorporate in thispoint> of my work that it isn't usefull at the moment.That's unfortunate to hear. We've tried to make it as dead simple as possible. The typical end-user Makefile for LLVM is about 3 or 4 lines long. What do you find confusing? Have you read the Makefile Guide?> I use the libraries directly > in my own Makefile. I tried to explicitly insert **all** LLVM librariesand> it doesn't work :-(Order of specification of the libraries on the link command is important. There are numerous inter-dependencies amongst the libraries and you need to understand them before this kind of approach will work. Its unlikely that you would want *all* LLVM libraries in any tool.> > Does anybody have got a example of an aplication which uses thesefunctions> to compile and execute in memory a multi-file application? (lli isn'tuseful> for me)The closest thing we have is the HowToUseJIT example program. I believe Duraid Madina is working on something similar. You might want to check with him (duraid at octopus dot com dot au)> > Thanks in advance and apologize so basic questions.Glad to help. I hope you find LLVM useful. Reid
On Fri, 2004-12-31 at 05:30, Francisco Puentes wrote:> Hi again, and thanks (Reid) for your fast response:Glad to help.> > Yes, it works!!! Only changing the order of libraries in the Makefile.Great!> > Nowaday I have my software with the capability of compile assembly, bytecode > (from buffer and file) and link them with a set of libraries. It seems to > work perfectly (I don't generate code yet).Cool. Next step: generate code! :)> My real aim is to have a process (host) with execute several no-jit binaries > (guest) each one in his own thread (not forked!! each one with a main > function). Guest and host have interdependencies in both ways. This is a > part of my doctoral degree. > > Before now, I was using TCC (Tiny C Compiler), but it have a big lack: it > isn't reentrant. It can only have one generated program at time. For this > reason I agree to use LLVM (two days ago) and rebuild all the project. 8-|Careful here. Much of LLVM is not re-entrant (yet!) either. However, if you're written your own execution engine and made it synchronize wisely, you should be okay. Making LLVM re-entrant is on the "to do" list, we just haven't had the time/manpower yet. If you find things in LLVM that would help make it more re-entrant, we'd gladly accept patches for it.> LLVM Makefile system is great!!:)> But I have no time to change nowadays all my > current Makefiles. Maybe later.Okay, sounds reasonable.> I have noticed that LLVM have memory leaks; exactly a poor main with a simple > "return 0;" linked with the LLVM libraries report memory laks using > valgrind. Is it normal?Yes, unfortunately it is. LLVM uses quite a few static variables because they are needed throughout the program's life span. They are constructed at static initialization time and never modified or freed so they show up as "leaks" in valgrind. Generally this isn't a problem. I would like to see this cleared up eventually too; its one of the stumbling blocks of making LLVM re-entrant as well. However, several important features of the LLVM Core (like the type system) depend on global variables. For example, a Type* for any given type is allocated exactly once so that types can be compared by pointer equality. To make that magic happen, we need to keep track of types globally and a static variable is currently used for that.> > Now I have other problem: I have a Module and I need generate a iostream > (memory) with native x86 code (maybe elf/coff) to be executed later (into > the guest process space, without fork!!). I studied llc and lli, but they > don't help me much. Any idea? Are there any guy working in some like that?There's two approaches that could be used here: 1. lli-style. Save the Module as bytecode and dynamically compile it at runtime. Your thread would: read the bytecode, convert it to x86 code in memory and then execute it directly from memory. 2. llc-style. Use llc to convert the module to assembly code, then use GCC (or as and ld with a little more effort) to create a dynamically loadable shared object (.so or .dll). You can then instantiate a DynamicLibrary object to load your compiled library into your thread (include/llvm/System/DynamicLibrary.h). Once loaded, your thread can use DynamicLibrary::GetAddressOfSymbol to find "main" and start executing it.> A sample code will be greatfully :-)Unfortunately, I don't know of any code that does this. bugpoint is close, but it uses fork(2). Reid. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20041231/f72fe936/attachment.sig>
On Fri, Dec 31, 2004 at 02:30:00PM +0100, Francisco Puentes wrote:> Now I have other problem: I have a Module and I need generate a > iostream (memory) with native x86 code (maybe elf/coff) to be executed > later (into the guest process space, without fork!!). I studied llc > and lli, but they don't help me much. Any idea? Are there any guy > working in some like that?In addition to what Reid said, I would like to point out that llvm/tools/lli/lli.cpp is not the entire JIT compiler, a lot of it lives in llvm/lib/ExecutionEngine/* and llvm/lib/ExecutionEngine/JIT/* . Together, they provide a lot of the machinery that you're looking for. The other point is that the JIT, when it writes out code to memory, is no longer relocatable, i.e. it's bound to the addresses where it's written, with all symbol references resolved. LLVM does support relocations, but if you need to save binary code and then load it in different address spaces, or write out elf/coff binaries directly, you will need to do some work such as creating a BinaryObjectCodeEmitter (to parallel the current MachineCodeEmitter) and properly write out the correct binary image, with relocations intact. -- Misha Brukman :: http://misha.brukman.net :: http://llvm.cs.uiuc.edu
Really JIT isn't my goal. I prefer use a native execution engine; and ok, I don't need save the generated Module so it ever lives in memory, so if LLVM doesn't generate relocatable code it's fine for me. About the reentrant lacks of LLVM, I can convert my own code - which build the Module - into a critical section so I think it is enough; but I need to know if several independent Modules can live in the same address space running at same time (I agree). Would be great if we append into the documentation several "patters" to show how perform with LLVM. It would accelerate the learn curve for beginners like me, avoiding basic errors and mistakes. If I reach a good level with LLVM I can make these. Have you a sample to generate code into memory? To move LLVM into a reentrant library we could move all global symbols into a structure which would be the LLVM context. All (ok, not all) constructor classes would have as first parameter this context, so it would be easy lock/unlock the context and allow critical sections in the code. Happy new year :-) ===========================================You can use some of this accounts: * fran at cic-online.net * fpuentes at udc.es fpuentes at acm.org fpuentes at edu.xunta.es fpuentes at hotmail.com =========================================== -----Mensaje original----- De: Misha Brukman [mailto:brukman at uiuc.edu] Enviado el: viernes, 31 de diciembre de 2004 19:20 Para: Francisco Puentes CC: 'LLVM Developers Mailing List' Asunto: Re: [LLVMdev] Primer with LLVM On Fri, Dec 31, 2004 at 02:30:00PM +0100, Francisco Puentes wrote:> Now I have other problem: I have a Module and I need generate a > iostream (memory) with native x86 code (maybe elf/coff) to be executed > later (into the guest process space, without fork!!). I studied llc > and lli, but they don't help me much. Any idea? Are there any guy > working in some like that?In addition to what Reid said, I would like to point out that llvm/tools/lli/lli.cpp is not the entire JIT compiler, a lot of it lives in llvm/lib/ExecutionEngine/* and llvm/lib/ExecutionEngine/JIT/* . Together, they provide a lot of the machinery that you're looking for. The other point is that the JIT, when it writes out code to memory, is no longer relocatable, i.e. it's bound to the addresses where it's written, with all symbol references resolved. LLVM does support relocations, but if you need to save binary code and then load it in different address spaces, or write out elf/coff binaries directly, you will need to do some work such as creating a BinaryObjectCodeEmitter (to parallel the current MachineCodeEmitter) and properly write out the correct binary image, with relocations intact. -- Misha Brukman :: http://misha.brukman.net :: http://llvm.cs.uiuc.edu
> > Now I have other problem: I have a Module and I need generate a iostream > > (memory) with native x86 code (maybe elf/coff) to be executed later > (into > > the guest process space, without fork!!). I studied llc and lli, but > they > > don't help me much. Any idea? Are there any guy working in some like > that? > > There's two approaches that could be used here: > > 1. lli-style. Save the Module as bytecode and dynamically compile it at > runtime. Your thread would: read the bytecode, convert it to x86 > code in memory and then execute it directly from memory. >Dear developers: What are the classes evolved in to convert bytecode to x86 code directly? Without Just In Time (all at once).