Luke Evans
2009-Mar-16 02:54 UTC
[LLVMdev] n00b question: From module/bitcode to Mach-O dylib file directly?
> Aaah. Wasn't quite sure what you were doing here. I'm not positive > what llvm can emit via the writers (there's support for all parts of > the file format), but it could be extended to write one out. I'm just > not quite sure why :) > > -ericWell, ultimately I'm curious about what it would take to port a JVM based language (http://openquark.org) to LLVM. A main motivator though is language/library/platform integration on the Mac (Cocoa, Objective-C, autozone). I'm not too interested in a JIT at this point, but rather native code generation. So far I can see a MachOWriter (with an "AddMachOWriter" in FileWriters.h) and it looks like I get to pass an appropriate TargetMachine to this. I'm wondering if this is expected to be sufficient (added to a pass manager) to be able to output a library on the Mac - whether or not it actually works. I only picked LLVM up yesterday, so I'm still trying to understand how the parts work together. With my current lack of orientation I have very little intuition as to whether I'm on the right track, and if it fails, how close (or otherwise) I might have been to getting something going. So I suppose I've been looking for hints at the vague 'shape' of the code I would expect to connect together to get the output I want, and while there are some nice samples for some parts of LLVM, I haven't found much to help learn how to emit native .o or .dylib (assuming this is possible). In the meantime I'll spend some time trying to bang some of the aforementioned pieces together and see if I can get it to do anything interesting with a minimal module. Thanks. -- lwe -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090315/4b6d0655/attachment.html>
Jon Harrop
2009-Mar-16 03:16 UTC
[LLVMdev] n00b question: From module/bitcode to Mach-O dylib file directly?
On Monday 16 March 2009 02:54:53 Luke Evans wrote:> > Aaah. Wasn't quite sure what you were doing here. I'm not positive > > what llvm can emit via the writers (there's support for all parts of > > the file format), but it could be extended to write one out. I'm just > > not quite sure why :) > > > > -eric > > Well, ultimately I'm curious about what it would take to port a JVM > based language (http://openquark.org) to LLVM.You might consider building on HLVM instead: http://hlvm.forge.ocamlcore.org/ HLVM provides everything from tuples and boxed types to a garbage collector. You just supply it with an AST, either incrementally for JIT compilation (e.g. from a REPL) or it can spit out bitcode to build an executable. -- Dr Jon Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?e
someguy
2009-Mar-16 07:27 UTC
[LLVMdev] n00b question: From module/bitcode to Mach-O dylib file directly?
Hey Luke, Unfortunately, the 'state of the art' is that llc only really supports emission of native assembly files (-filetype=asm) which can then be assembled and linked with your native gas/ld. There is some source support for 'object file generation', exposed via the -filetype=obj flag to llc, but it is not complete, and totally broken in some cases. This is something I am trying to work on with Aaron. You can follow our discussion on the list here, and feel free to pitch in. That said, the MachO generation _should_ work (in 2.5) for outputting .o files, which would still need to be linked using your native ld into a dynlib. I don't see that llc will ever generate a dynlib, as I think that that is not its function. The most you can expect is a valid target object file. If you encounter a particular issue using the -filetype=obj flag, please let us know so we can fix it... I for one would really like to see object generation become a fully working feature of the llvm toolchain. On Mon, Mar 16, 2009 at 4:54 AM, Luke Evans <luke at eversosoft.com> wrote:> Aaah. Wasn't quite sure what you were doing here. I'm not positive > what llvm can emit via the writers (there's support for all parts of > the file format), but it could be extended to write one out. I'm just > not quite sure why :) > -eric > > Well, ultimately I'm curious about what it would take to port a JVM based > language (http://openquark.org) to LLVM. > A main motivator though is language/library/platform integration on the Mac > (Cocoa, Objective-C, autozone). I'm not too interested in a JIT at this > point, but rather native code generation. > So far I can see a MachOWriter (with an "AddMachOWriter" in FileWriters.h) > and it looks like I get to pass an appropriate TargetMachine to this. > I'm wondering if this is expected to be sufficient (added to a pass manager) > to be able to output a library on the Mac - whether or not it actually > works. > I only picked LLVM up yesterday, so I'm still trying to understand how the > parts work together. With my current lack of orientation I have very little > intuition as to whether I'm on the right track, and if it fails, how close > (or otherwise) I might have been to getting something going. So I suppose > I've > been looking for hints at the vague 'shape' of the code I would expect to > connect together to get the output I want, and while there are some nice > samples for some parts of LLVM, I haven't found much to help learn how to > emit native .o or .dylib (assuming this is possible). > In the meantime I'll spend some time trying to bang some of the > aforementioned pieces together and see if I can get it to do anything > interesting with a minimal module. > Thanks. > -- lwe > > > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >
Devang Patel
2009-Mar-16 18:30 UTC
[LLVMdev] n00b question: From module/bitcode to Mach-O dylib file directly?
On Mar 15, 2009, at 7:54 PM, Luke Evans wrote:> So far I can see a MachOWriter (with an "AddMachOWriter" in > FileWriters.h) and it looks like I get to pass an appropriate > TargetMachine to this. > I'm wondering if this is expected to be sufficient (added to a pass > manager) to be able to output a library on the Mac - whether or not > it actually works.IIUC, MachOWriter is not yet 100% complete. It is a work in progress, which is not getting significant attention and love recently. It'd be very useful to have this completed. - Devang
Luke Evans
2009-Mar-16 20:51 UTC
[LLVMdev] Bit-code file input to ld (was: ...From module/bitcode to Mach-O dylib file directly)
> It is simpler. There is not any Mach-O envelope. The platform linker > can directly read Mach-O files as well as llvm bit-code file (using > llvm bit-code file reader). > > > Not sure if/where the exact form of the Mach-O file that carries bit > > code is documented.Thanks for that. Yes, I see what you mean (no Mach-O envelope), yet there seems to be something fundamentally different between a bit-code file I create from Apple's llvm-gcc-4.2, and one created from the LLVM APIs. Particularly: - ld won't consume the/a bit-code file that is directly generated by the LLVM APIs: "ld warning: in foo.bc, file is not of required architecture" - The bit-code file I have from LLVM differs from ones generated by Apple's llvm-gcc-4.2. The latter has a "DE C0 17 0B" magic introducer, and there seems to be another 16 bytes before the "BC Code" (42 43 C0 DE) magic appears. Whereas, the file generated from the LLVM APIs starts immediately with the "BC Code" magic number. There's a really good chance that this is just a case of 'user error'. At the very least it seems to me that I'm not doing something that's required to emit bit-code files in the format that they are consumable by ld. Alternatively, there is some kind of wrapper that the llvm-gcc-4.2 tool produces around the basic bit-code, though not as you've pointed out, a Mach-O format. FWIW, here are the first 24 bytes, up to and including what I recognise as the LLVM bit-code magic number: DE C0 17 0B 00 00 00 00 14 00 00 00 AC 01 00 00 07 00 00 00 42 43 C0 DE... The 'file' util identifies this as "Compiled PSI (v1) data", though that's probably not relevant/useful, as it just comes from the DE C0, which could be any number of things. Anyway, I guess my simple question is: What do I need to do to get home-brew bit-code output from the LLVM bit-code writer to conform to whatever requirements that ld has for input? -- lwe -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090316/8146a964/attachment.html>
Devang Patel
2009-Mar-16 21:14 UTC
[LLVMdev] Bit-code file input to ld (was: ...From module/bitcode to Mach-O dylib file directly)
Hi Luke, On Mar 16, 2009, at 1:51 PM, Luke Evans wrote:>> It is simpler. There is not any Mach-O envelope. The platform linker >> can directly read Mach-O files as well as llvm bit-code file (using >> llvm bit-code file reader). >> >> > Not sure if/where the exact form of the Mach-O file that carries >> bit >> > code is documented. > > > Thanks for that. Yes, I see what you mean (no Mach-O envelope), yet > there seems to be something fundamentally different between a bit- > code file I create from Apple's llvm-gcc-4.2, and one created from > the LLVM APIs. > Particularly: > - ld won't consume the/a bit-code file that is directly generated by > the LLVM APIs: "ld warning: in foo.bc, file is not of required > architecture" > - The bit-code file I have from LLVM differs from ones generated by > Apple's llvm-gcc-4.2. The latter has a "DE C0 17 0B" magic > introducer, and there seems to be another 16 bytes before the "BC > Code" (42 43 C0 DE) magic appears. > Whereas, the file generated from the LLVM APIs starts immediately > with the "BC Code" magic number.Yes. If you set the target as darwin for the module while generating bit-code file from the LLVM APIs then llvm bit-code writer will add this. See llvm/lib/Bitcode/Writer/BitcodeWriter.cpp for more info. 1292 /// EmitDarwinBCHeader - If generating a bc file on darwin, we have to emit a 1293 /// header and trailer to make it compatible with the system archiver. To do 1294 /// this we emit the following header, and then emit a trailer that pads the 1295 /// file out to be a multiple of 16 bytes. 1296 /// 1297 /// struct bc_header { 1298 /// uint32_t Magic; // 0x0B17C0DE 1299 /// uint32_t Version; // Version, currently always 0. 1300 /// uint32_t BitcodeOffset; // Offset to traditional bitcode file. 1301 /// uint32_t BitcodeSize; // Size of traditional bitcode file. 1302 /// uint32_t CPUType; // CPU specifier. 1303 /// ... potentially more later ... 1304 /// }; 1305 enum { 1306 DarwinBCSizeFieldOffset = 3*4, // Offset to bitcode_size. 1307 DarwinBCHeaderSize = 5*4 1308 }; 1309 and 1388 /// WriteBitcodeToStream - Write the specified module to the specified output 1389 /// stream. 1390 void llvm::WriteBitcodeToStream(const Module *M, BitstreamWriter &Stream) { 1391 // If this is darwin, emit a file header and trailer if needed. 1392 bool isDarwin = M->getTargetTriple().find("-darwin") != std::string::npos; 1393 if (isDarwin) 1394 EmitDarwinBCHeader(Stream, M->getTargetTriple()); 1395> > There's a really good chance that this is just a case of 'user > error'. At the very least it seems to me that I'm not doing > something that's required to emit bit-code files in the format that > they are consumable by ld. Alternatively, there is some kind of > wrapper that the llvm-gcc-4.2 tool produces around the basic bit- > code, though not as you've pointed out, a Mach-O format. > > FWIW, here are the first 24 bytes, up to and including what I > recognise as the LLVM bit-code magic number: > DE C0 17 0B 00 00 00 00 14 00 00 00 AC 01 00 00 07 00 00 > 00 42 43 C0 DE... > > The 'file' util identifies this as "Compiled PSI (v1) data", though > that's probably not relevant/useful, as it just comes from the DE > C0, which could be any number of things. > > Anyway, I guess my simple question is: > What do I need to do to get home-brew bit-code output from the LLVM > bit-code writer to conform to whatever requirements that ld has for > input? > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090316/789ef502/attachment.html>
Luke Evans
2009-Mar-17 04:50 UTC
[LLVMdev] Bit-code file input to ld (was: ...From module/bitcode to Mach-O dylib file directly)
> Anyway, I should be able to figure it out from here - thanks again....which I did - though I had to ensure I had the target triple set in the module just right: "x86_64-apple-darwin" Originally I tried "i686-apple-darwin9" as this is what gcc reports, and what gets burnt into config.h as: #define LLVM_HOSTTRIPLE "i686-apple-darwin9.6.0" However, it dawned on me that I should really be asserting x86_64, and indeed this works nicely if it is also asserted on the ld command line. -- lwe
Apparently Analagous Threads
- [LLVMdev] n00b question: From module/bitcode to Mach-O dylib file directly?
- [LLVMdev] Bit-code file input to ld (was: ...From module/bitcode to Mach-O dylib file directly)
- [LLVMdev] Bit-code file input to ld (was: ...From module/bitcode to Mach-O dylib file directly)
- [LLVMdev] n00b question: From module/bitcode to Mach-O dylib file directly?
- [LLVMdev] n00b question: From module/bitcode to Mach-O dylib file directly?