Hi Nate, I've successfully ported one bitcode from Linux to Mac to Windows. All were x86 and the program was text-based, but I'd say my LLVM Wrapper would be worth some effort in the future if I could just get some help. Currently it just wraps StdIO.h with its own functions. Here's some of what it would take to make portable bitcodes in C or LLVM Assembly: * Convert all preprocessor conditionals to regular conditionals so that both the #ifdef parts and the #else parts make it into the bitcode. Don't worry about bloat since the installer will be able to run constant folding and dead-code elimination to get rid of the unused parts. The constants will be supplied by the installer in the form of an external bitcode for linkage. The only exception to this rule would be the guard-defines in .h files. * Make sure that the code in the distribution bitcode is endian-agnostic 32-bit code. A sandbox will be needed on 64-bit systems or else a separate 64-bit bitcode package will still be required. * Make sure all external dependencies including LibC runtimes are accessed from the wrapper rather than the native code from the bitcode. All external libraries are system-specific. There are probably other requirements that I've either overlooked or forgotten but I'd like to have my wrapper expanded to do some more extensive functionality if somebody would like to join the project, let me know and I'll add you to the SourceForge project at http://sourceforge.net/projects/llvmlibc/ . --Sam ----- Original Message -----> From: Nate Fries <nfries88 at yahoo.com> > To: LLVM Developers Mailing List <llvmdev at cs.uiuc.edu> > Cc: > Sent: Wednesday, June 1, 2011 4:25 PM > Subject: Re: [LLVMdev] Thinking about "whacky" backends > > Cameron Zwarich wrote: > >> What benefit do you get from having a backend here rather than an > interpreter for LLVM IR? > The same thing as an interpreter, just a native build (no need for an > interpreter program, better speed, etc). > > This would be beneficial anywhere that "build once, deploy anywhere" > functionality is desired, without resorting to using a higher-level language > like C# or Java. > Granted, the application that either interprets or compiles and links the > resulting bitcode application would still be required on the system, much like a > VM for this language; but for developers not familiar with such languages and > either no particular desire or very little time to become familiar with them, > this would be an excellent solution. > > That said, it seems like it ought to be possible to do the same thing by > emitting bitcode for all supported platform/arch combinations and compressing > them in an archive, then decompressing and either interpreting or JIT-compiling > the appropriate bitcode for the platform. This would just be a more flexible > means to that same end. > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >
On 6/2/2011 10:13 AM, Samuel Crow wrote:> Hi Nate, > > I've successfully ported one bitcode from Linux to Mac to Windows. All were x86 and the program was text-based, but I'd say my LLVM Wrapper would be worth some effort in the future if I could just get some help. Currently it just wraps StdIO.h with its own functionsNaturally that would work perfectly fine on a similar architecture and common dependencies. I'm a hobbyist game developer, so that is my primary concern. Even using a cross-platform game library like Irrlicht or Ogre, there can be problems with using the same bitcode for each platform. Especially in cases where you may have to implement system-specific code to cover cases not provided for by such libraries (wrappers for MessageBox/NSRunAlertPanel, Clipboard/Pasteboard, etc)> * Convert all preprocessor conditionals to regular conditionals so that both the #ifdef parts and the #else parts make it into the bitcode. Don't worry about bloat since the installer will be able to run constant folding and dead-code elimination to get rid of the unused parts. The constants will be supplied by the installer in the form of an external bitcode for linkage. The only exception to this rule would be the guard-defines in .h files.I'm a huge fan of the C preprocessor, and this just seems like a terrible idea. Better idea would be to generate some sort of metadata from these #ifdef's instead.> * Make sure that the code in the distribution bitcode is endian-agnostic 32-bit code. A sandbox will be needed on 64-bit systems or else a separate 64-bit bitcode package will still be required. >Endian agnosticism without using the preprocessor just seems... burdensome. However, if I'm grabbing the correct information here, you're saying that bitcode for x86 Linux would function on 32-bit ARM, PPC, and SPARC Linux (if endian-agnostic)?> * Make sure all external dependencies including LibC runtimes are accessed from the wrapper rather than the native code from the bitcode. All external libraries are system-specific.The meaning here being that I need all dependencies to be part of the bitcode, as opposed to being natively linked later? This simply isn't practical for games.
----- Original Message -----> From: Nate Fries <nfries88 at yahoo.com> > To: LLVM Developers Mailing List <llvmdev at cs.uiuc.edu> > Cc: > Sent: Thursday, June 2, 2011 3:26 PM > Subject: Re: [LLVMdev] Thinking about "whacky" backends > > On 6/2/2011 10:13 AM, Samuel Crow wrote: >> Hi Nate, >> >> I've successfully ported one bitcode from Linux to Mac to Windows. All > were x86 and the program was text-based, but I'd say my LLVM Wrapper would > be worth some effort in the future if I could just get some help. Currently it > just wraps StdIO.h with its own functions > Naturally that would work perfectly fine on a similar architecture and > common dependencies. > I'm a hobbyist game developer, so that is my primary concern. Even using > a cross-platform game library like Irrlicht or Ogre, there can be > problems with using the same bitcode for each platform. Especially in > cases where you may have to implement system-specific code to cover > cases not provided for by such libraries (wrappers for > MessageBox/NSRunAlertPanel, Clipboard/Pasteboard, etc) >> * Convert all preprocessor conditionals to regular conditionals so that > both the #ifdef parts and the #else parts make it into the bitcode. Don't > worry about bloat since the installer will be able to run constant folding and > dead-code elimination to get rid of the unused parts. The constants will be > supplied by the installer in the form of an external bitcode for linkage. The > only exception to this rule would be the guard-defines in .h files. > I'm a huge fan of the C preprocessor, and this just seems like a > terrible idea. > Better idea would be to generate some sort of metadata from these > #ifdef's instead. >> * Make sure that the code in the distribution bitcode is endian-agnostic > 32-bit code. A sandbox will be needed on 64-bit systems or else a separate > 64-bit bitcode package will still be required. >> > Endian agnosticism without using the preprocessor just seems... burdensome. > However, if I'm grabbing the correct information here, you're saying > that bitcode for x86 Linux would function on 32-bit ARM, PPC, and SPARC > Linux (if endian-agnostic)? >> * Make sure all external dependencies including LibC runtimes are accessed > from the wrapper rather than the native code from the bitcode. All external > libraries are system-specific. > The meaning here being that I need all dependencies to be part of the > bitcode, as opposed to being natively linked later? This simply isn't > practical for games.Hello again Nate, Fixing preprocessor macros is simple: #ifdef __windows // windows code here #else // other code here #endif becomes extern const bool isWindows; // note: I'm not sure if const is legal here but it should be if (isWindows) { //windows code here } else { //other code here } And isWindows will be defined in the wrapper to allow the external symbol to be resolved at final link time. The point here is that if only the Windows code was in the main bitcode, there would be no way to get the other code to link to the other OSs since the preprocessor would have killed the other code before it even got compiled. The same technique can be applied to endianness switching. To make a cross-platform library such as Irrlicht work on more than one platform, Irrlicht would need a wrapper as well. This is turning into a lot of work now, as you can see. This is why I'm asking for help. The rewards are great however. It means that you can compile once on Windows and install that bitcode on Windows, Linux, Mac, FreeBSD, etc. If you followed the non-preprocessor if statements technique for your endianness swapping, it should work on PPC, Intel, ARM, SPARC and any other LLVM-supported 32-bit processor. Only the wrapper would need to be system-specific. Does this make sense? Please keep writing if you have any additional questions! I think we're on a roll here. --Sam
Samuel Crow <samuraileumas at yahoo.com> writes:> Here's some of what it would take to make portable bitcodes in C or LLVM Assembly:A look at the work done on ANDF in the 90's may be helpful. I've only skimmed it but there's been some deep thinking about stuff like this. -Dave
----- Original Message -----> From: David A. Greene <greened at obbligato.org> > To: Samuel Crow <samuraileumas at yahoo.com> > Cc: Nate Fries <nfries88 at yahoo.com>; LLVM Developers Mailing List <llvmdev at cs.uiuc.edu> > Sent: Friday, June 3, 2011 4:38 PM > Subject: Re: [LLVMdev] Thinking about "whacky" backends > > Samuel Crow <samuraileumas at yahoo.com> writes: > >> Here's some of what it would take to make portable bitcodes in C or > LLVM Assembly: > > A look at the work done on ANDF in the 90's may be helpful. I've only > skimmed it but there's been some deep thinking about stuff like this. > > -Dave >Thanks Dave, A little bit of Google reveals a wikipedia article: http://en.wikipedia.org/wiki/Architecture_Neutral_Distribution_Format that links to http://en.wikipedia.org/wiki/TenDRA_Distribution_Format which has its specification listed at http://docs.tendra.org/reference/xhtml/guide which speaks in lofty abstract language terms and doesn't appear to be a binary representation of code. I'm not sure if any of this actually helps us considering that LLVM already has mechanisms for most of these functions. Here we go again, --Sam