llvm dev - May 2010 - [LLVMdev] Using LLVM to compile Objective-C on an Xbox 360

We are looking at using Objective-C/C++ in a new game engine.  Objective C's
duality of being both very dynamic and very "C" gives us exactly what
we need to make the SDK and engineering of games simpler.

This means that we will need a way to compile it on all platforms our games will
target.  Currently the major platforms we are concerned with include... PC, Mac,
XBox 360, PS3, iPhone.  Now the PC, Mac, iPhone and PS3 are fairly simple.  If
we build our own OpenStep libraries we can simply use LLVM to compile directly
to these platforms; the PS3 although proprietary uses GCC as a C/C++ compiler so
I am assuming Objective-C can be used fairly simply.  This leaves us with the
XBox 360.

The 360 is a special chip (PowerPC based) with, as far as I have researched, a
special ABI (Windows derivative).  I haven't the faintest clue of whether
code from the LLVM PPC backend would even work on the 360, much less
interoperate with the system libraries. So my formulated solution has become
this: use an LLVM backend to output C code and then compile that code with using
MS's XBox 360 compiler. I believe I have read that LLVM has a C backend
already but I don't know how to select it.

If I can get a proof of concept showing Objective-C code running on the 360 we
are off to the races.  Any help is appreciated just not sure if all the
pieces/parts exist and/or what I am missing.  So... is this feasible?  If so...
how do I get LLVM to output C code?

Thanks,
Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20100526/b3c82db8/attachment.html>

llvm can output C code, but that target has bitrotted severely over the last few
months and nobody seems to be interested in fixing it.  You may need to do some
work there.  Alternatively you could implement the PPC ABI that you need.  There
are several examples of supporting multiple ABIs on the same hardware, x86 being
the most obvious.  A lot of simple stuff will probably Just Work with the
existing PPC ABI, more complicated stuff may not.  (Only 32-bit PPC is really
maintained, though, there are probably lots of problems with 64-bit.)

Objective C in llvm, AFAIK, is only used on the MacOSX targets and only tested
there.   There are sufficient secret handshakes between the compiler and the
ObjC runtime that it is unlikely to Just Work in an untested environment. 
OpenStep has a familial relationship to MacOSX ObjC runtime, but they aren't
the same and are unlikely to still be binary compatible at this point.  You may
need to do some work there also.

Summary, you can probably get this approach to work, but it's not as easy as
you're hoping.

On May 26, 2010, at 5:20 PMPDT, kdubb wrote:
> We are looking at using Objective-C/C++ in a new game engine.  Objective
C's duality of being both very dynamic and very "C" gives us
exactly what we need to make the SDK and engineering of games simpler.
> 
> This means that we will need a way to compile it on all platforms our games
will target.  Currently the major platforms we are concerned with include... PC,
Mac, XBox 360, PS3, iPhone.  Now the PC, Mac, iPhone and PS3 are fairly simple. 
If we build our own OpenStep libraries we can simply use LLVM to compile
directly to these platforms; the PS3 although proprietary uses GCC as a C/C++
compiler so I am assuming Objective-C can be used fairly simply.  This leaves us
with the XBox 360.
> 
> The 360 is a special chip (PowerPC based) with, as far as I have
researched, a special ABI (Windows derivative).  I haven't the faintest clue
of whether code from the LLVM PPC backend would even work on the 360, much less
interoperate with the system libraries. So my formulated solution has become
this: use an LLVM backend to output C code and then compile that code with using
MS's XBox 360 compiler. I believe I have read that LLVM has a C backend
already but I don't know how to select it.
> 
> If I can get a proof of concept showing Objective-C code running on the 360
we are off to the races.  Any help is appreciated just not sure if all the
pieces/parts exist and/or what I am missing.  So... is this feasible?  If so...
how do I get LLVM to output C code?
> 
> Thanks,
> Kevin
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

I'm slightly confused here, just to be clear: you're planning to go ObjC
-> LLVM (via Clang or whatever) -> C, then use the 360 SDK compiler to
build that, right? If that's the case, I think it'd be a better option
than to use LLVM to directly generate XEXs, and you won't give the TCR guys
fits (I checked the BAS TCRs, none of them say you can't do that, but the
platform has some requirements regarding binary code that I doubt you'll get
right without a lot of effort). If that is the pipeline you had in mind, have
you attempted to run a Hello World app yet? Ran into any errors?

- Sherief

On May 26, 2010, at 8:20 PM, kdubb wrote:
> We are looking at using Objective-C/C++ in a new game engine.  Objective
C's duality of being both very dynamic and very "C" gives us
exactly what we need to make the SDK and engineering of games simpler.
> 
> This means that we will need a way to compile it on all platforms our games
will target.  Currently the major platforms we are concerned with include... PC,
Mac, XBox 360, PS3, iPhone.  Now the PC, Mac, iPhone and PS3 are fairly simple. 
If we build our own OpenStep libraries we can simply use LLVM to compile
directly to these platforms; the PS3 although proprietary uses GCC as a C/C++
compiler so I am assuming Objective-C can be used fairly simply.  This leaves us
with the XBox 360.
> 
> The 360 is a special chip (PowerPC based) with, as far as I have
researched, a special ABI (Windows derivative).  I haven't the faintest clue
of whether code from the LLVM PPC backend would even work on the 360, much less
interoperate with the system libraries. So my formulated solution has become
this: use an LLVM backend to output C code and then compile that code with using
MS's XBox 360 compiler. I believe I have read that LLVM has a C backend
already but I don't know how to select it.
> 
> If I can get a proof of concept showing Objective-C code running on the 360
we are off to the races.  Any help is appreciated just not sure if all the
pieces/parts exist and/or what I am missing.  So... is this feasible?  If so...
how do I get LLVM to output C code?
> 
> Thanks,
> Kevin
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

This is certainly an option... but keeping up an LLVM backend for the 360 seems
like a bit much work.  Using Clang's rewriter requires no real low-level
maintenance, just creating and maintaining a C library for support functions
emitted by the compiler (e.g. objc_msgSend).

It seems to create a backend for LLVM targeting the 360 I wouldn't need to
create something that outputs XEX's anyway. My custom backend would
effectively create object files that the 360's linker would have to be able
to recognize.  This would mean not only outputting all the correct ABI
conforming code but the correct PDB debugging information as well.  It seems
much less, and easier, work to implement the support functions and run it
through the standard xbox compiler as the ABI and debugging information are
taken care of.

On May 26, 2010, at 8:08 PM, Sherief Farouk wrote:
> I'm slightly confused here, just to be clear: you're planning to go
ObjC -> LLVM (via Clang or whatever) -> C, then use the 360 SDK compiler
to build that, right? If that's the case, I think it'd be a better
option than to use LLVM to directly generate XEXs, and you won't give the
TCR guys fits (I checked the BAS TCRs, none of them say you can't do that,
but the platform has some requirements regarding binary code that I doubt
you'll get right without a lot of effort). If that is the pipeline you had
in mind, have you attempted to run a Hello World app yet? Ran into any errors?
> 
> - Sherief
> 
> On May 26, 2010, at 8:20 PM, kdubb wrote:
> 
>> We are looking at using Objective-C/C++ in a new game engine. 
Objective C's duality of being both very dynamic and very "C"
gives us exactly what we need to make the SDK and engineering of games simpler.
>> 
>> This means that we will need a way to compile it on all platforms our
games will target.  Currently the major platforms we are concerned with
include... PC, Mac, XBox 360, PS3, iPhone.  Now the PC, Mac, iPhone and PS3 are
fairly simple.  If we build our own OpenStep libraries we can simply use LLVM to
compile directly to these platforms; the PS3 although proprietary uses GCC as a
C/C++ compiler so I am assuming Objective-C can be used fairly simply.  This
leaves us with the XBox 360.
>> 
>> The 360 is a special chip (PowerPC based) with, as far as I have
researched, a special ABI (Windows derivative).  I haven't the faintest clue
of whether code from the LLVM PPC backend would even work on the 360, much less
interoperate with the system libraries. So my formulated solution has become
this: use an LLVM backend to output C code and then compile that code with using
MS's XBox 360 compiler. I believe I have read that LLVM has a C backend
already but I don't know how to select it.
>> 
>> If I can get a proof of concept showing Objective-C code running on the
360 we are off to the races.  Any help is appreciated just not sure if all the
pieces/parts exist and/or what I am missing.  So... is this feasible?  If so...
how do I get LLVM to output C code?
>> 
>> Thanks,
>> Kevin
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Implementing the backend (or editing the current PPC backend as needed) is a
definite option.  This seems to be the real question... which is easier...
maintaining the PPC backend or maintaining the rewriter.  Currently (in
admittedly trivial tests) I have gotten the rewriter to work and output C code. 
There are some outstanding issues to do with linking and accessing the
reflection information objective-c provides (and the rewriter properly outputs)
but I am sure the person/people who wrote the rewriter can answer those fairly
simply (or else the rewriter would be fairly useless).

As far as the runtime goes there is an implementation for windows, linux and
most likely the PS3 (since it is already a linux derivative using GCC as
mentioned).  If we have to create a runtime for PS3 and XBox that seems trivial
as the functions are very basic in nature.

My comments on OpenStep were more meant to point to the fact that we would write
our own library; really just throwing all notion of OpenStep away.  Instead of
NSObject we would create our own base DObject or something along with a DString,
DAarry, DSet, DMap, etc.  Truthfully this would be our plan anyway because we
want to follow the lead of OSX and provide these objects "toll free
bridged"... meaning we would implement them using a std c++ library object
(e.g. std::basic_string or std::vector) as the first member of the object which
will allow us to cast a DString to std::string or DArray to std::vector without
any manual or automatic marshaling. As mentioned this is how OSX implements its
NS classes, by using the equivalent CF version which again allows casting
between the objects.

On May 26, 2010, at 6:19 PM, Dale Johannesen wrote:
> llvm can output C code, but that target has bitrotted severely over the
last few months and nobody seems to be interested in fixing it.  You may need to
do some work there.  Alternatively you could implement the PPC ABI that you
need.  There are several examples of supporting multiple ABIs on the same
hardware, x86 being the most obvious.  A lot of simple stuff will probably Just
Work with the existing PPC ABI, more complicated stuff may not.  (Only 32-bit
PPC is really maintained, though, there are probably lots of problems with
64-bit.)
> 
> Objective C in llvm, AFAIK, is only used on the MacOSX targets and only
tested there.   There are sufficient secret handshakes between the compiler and
the ObjC runtime that it is unlikely to Just Work in an untested environment. 
OpenStep has a familial relationship to MacOSX ObjC runtime, but they aren't
the same and are unlikely to still be binary compatible at this point.  You may
need to do some work there also.
> 
> Summary, you can probably get this approach to work, but it's not as
easy as you're hoping.
> 
> On May 26, 2010, at 5:20 PMPDT, kdubb wrote:
> 
>> We are looking at using Objective-C/C++ in a new game engine. 
Objective C's duality of being both very dynamic and very "C"
gives us exactly what we need to make the SDK and engineering of games simpler.
>> 
>> This means that we will need a way to compile it on all platforms our
games will target.  Currently the major platforms we are concerned with
include... PC, Mac, XBox 360, PS3, iPhone.  Now the PC, Mac, iPhone and PS3 are
fairly simple.  If we build our own OpenStep libraries we can simply use LLVM to
compile directly to these platforms; the PS3 although proprietary uses GCC as a
C/C++ compiler so I am assuming Objective-C can be used fairly simply.  This
leaves us with the XBox 360.
>> 
>> The 360 is a special chip (PowerPC based) with, as far as I have
researched, a special ABI (Windows derivative).  I haven't the faintest clue
of whether code from the LLVM PPC backend would even work on the 360, much less
interoperate with the system libraries. So my formulated solution has become
this: use an LLVM backend to output C code and then compile that code with using
MS's XBox 360 compiler. I believe I have read that LLVM has a C backend
already but I don't know how to select it.
>> 
>> If I can get a proof of concept showing Objective-C code running on the
360 we are off to the races.  Any help is appreciated just not sure if all the
pieces/parts exist and/or what I am missing.  So... is this feasible?  If so...
how do I get LLVM to output C code?
>> 
>> Thanks,
>> Kevin
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>