llvm dev - May 2010 - [LLVMdev] Using LLVM to compile Objective-C on an Xbox 360

Yep... I have been a 360 developer since the days when it was a actual Macintosh
G5.  I went to all the early tech. seminars and learned way more about that
wacky processor and MS's seemingly crazy ABI choices than I care to (I know,
I know, they had their reasons for them).  Knowing this is why I quickly turned
toward rewriting objc instead of attempting to implement a backend.

I have also had a PS3 dev kit since the days it was a big grey can of
fire-hazard.  I am much better versed in the 360 but it seems Sony made many
more sane decisions to build on existing technology.  Although, it sounds like
Alex is alluding to the fact that I may be wrong on just how much they built on
and how much they rewrote; on the surface though trivial integration is
certainly possible between generic GCC and PS3 tools. I will get on DevNet later
today and discuss that with him.

Overall, as we both have said before, the ObjC => C route seems like the best
idea.  It just means that we need to build/maintain our own objective-c runtime
for those platforms that require them and make sure to keep the
"-rewrite-objc" option in clang maintained and ensure it produces
workable code for all platforms (their is currently some very GCC specific code
being output).  This allows each proprietary platform to generate code in their
own fashion and it really is a minor extra step to first run our code through
clang.

To ease our development process, actually hide it completely, we are planning on
leveraging the open nature of Clang and LLVM to create a driver for these
platforms that first rewrites ObjC to C and then calls the platform's C
compiler passing through all command line options. This way the ObjC stage
should be fairly transparent.

On May 27, 2010, at 11:15 AM, Sherief Farouk wrote:
> Kevin, there're some unwritten rules on the 360 that seem to interfere
with code generation - pointer load-stores seem to require zeroing the most
significant 32 bits, for example - that was one issue that I ran into a while
ago while fooling around with code generation on the 360 (and don't blame me
if I'm slightly off the mark, it was quite a while ago). I didn't seem
to find it mentioned in the XDK docs anywhere. Again, you'll be better off
generating C code and using the XDK compiler to compile that.
> 
> - Sherief
> 
> On May 27, 2010, at 12:26 PM, Kevin Wooten wrote:
> 
>> By linux derivative I meant that it borrows the linux GCC ABI... and it
does.  You can compile with an off the shelf GCC cross compiler and link the
resultant object files ones compiled with the PS3 provided version.   We have
done it.
>> 
>> Also, as both an XBox 360 and PS3 developer, there seems to me to be
nothing in the TCRs/TRCs that preclude us from using a different compiler. 
There are rules about symbol inclusion and other resultant binary
requirements... but as of yet I have not found specific ones stopping us from
using a compiler that works.  In either case the LLVM rewriter solves any other
these issues as we would be just be compiling C.
>> 
>> On May 27, 2010, at 8:09 AM, Alex Rosenberg wrote:
>> 
>>> PS3 is not "a Linux derivative." The compilers supplied
by SCE for PS3 game development are highly customized and support a customized
ABI that will take some time to adjust LLVM and Clang to support.
>>> 
>>> You'd likely also run afoul of a TRC or two, similar to the
problems you'll face with Microsoft TCRs mentioned earlier.
>>> 
>>> Alex
>>> 
>>> On May 27, 2010, at 12:15 AM, Kevin Wooten wrote:
>>> 
>>>> Implementing the backend (or editing the current PPC backend as
needed) is a definite option.  This seems to be the real question... which is
easier... maintaining the PPC backend or maintaining the rewriter.  Currently
(in admittedly trivial tests) I have gotten the rewriter to work and output C
code.  There are some outstanding issues to do with linking and accessing the
reflection information objective-c provides (and the rewriter properly outputs)
but I am sure the person/people who wrote the rewriter can answer those fairly
simply (or else the rewriter would be fairly useless).
>>>> 
>>>> As far as the runtime goes there is an implementation for
windows, linux and most likely the PS3 (since it is already a linux derivative
using GCC as mentioned).  If we have to create a runtime for PS3 and XBox that
seems trivial as the functions are very basic in nature.
>>>> 
>>>> My comments on OpenStep were more meant to point to the fact
that we would write our own library; really just throwing all notion of OpenStep
away.  Instead of NSObject we would create our own base DObject or something
along with a DString, DAarry, DSet, DMap, etc.  Truthfully this would be our
plan anyway because we want to follow the lead of OSX and provide these objects
"toll free bridged"... meaning we would implement them using a std c++
library object (e.g. std::basic_string or std::vector) as the first member of
the object which will allow us to cast a DString to std::string or DArray to
std::vector without any manual or automatic marshaling. As mentioned this is how
OSX implements its NS classes, by using the equivalent CF version which again
allows casting between the objects.
>>>> 
>>>> On May 26, 2010, at 6:19 PM, Dale Johannesen wrote:
>>>> 
>>>>> llvm can output C code, but that target has bitrotted
severely over the last few months and nobody seems to be interested in fixing
it.  You may need to do some work there.  Alternatively you could implement the
PPC ABI that you need.  There are several examples of supporting multiple ABIs
on the same hardware, x86 being the most obvious.  A lot of simple stuff will
probably Just Work with the existing PPC ABI, more complicated stuff may not. 
(Only 32-bit PPC is really maintained, though, there are probably lots of
problems with 64-bit.)
>>>>> 
>>>>> Objective C in llvm, AFAIK, is only used on the MacOSX
targets and only tested there.   There are sufficient secret handshakes between
the compiler and the ObjC runtime that it is unlikely to Just Work in an
untested environment.  OpenStep has a familial relationship to MacOSX ObjC
runtime, but they aren't the same and are unlikely to still be binary
compatible at this point.  You may need to do some work there also.
>>>>> 
>>>>> Summary, you can probably get this approach to work, but
it's not as easy as you're hoping.
>>>>> 
>>>>> On May 26, 2010, at 5:20 PMPDT, kdubb wrote:
>>>>> 
>>>>>> We are looking at using Objective-C/C++ in a new game
engine.  Objective C's duality of being both very dynamic and very
"C" gives us exactly what we need to make the SDK and engineering of
games simpler.
>>>>>> 
>>>>>> This means that we will need a way to compile it on all
platforms our games will target.  Currently the major platforms we are concerned
with include... PC, Mac, XBox 360, PS3, iPhone.  Now the PC, Mac, iPhone and PS3
are fairly simple.  If we build our own OpenStep libraries we can simply use
LLVM to compile directly to these platforms; the PS3 although proprietary uses
GCC as a C/C++ compiler so I am assuming Objective-C can be used fairly simply. 
This leaves us with the XBox 360.
>>>>>> 
>>>>>> The 360 is a special chip (PowerPC based) with, as far
as I have researched, a special ABI (Windows derivative).  I haven't the
faintest clue of whether code from the LLVM PPC backend would even work on the
360, much less interoperate with the system libraries. So my formulated solution
has become this: use an LLVM backend to output C code and then compile that code
with using MS's XBox 360 compiler. I believe I have read that LLVM has a C
backend already but I don't know how to select it.
>>>>>> 
>>>>>> If I can get a proof of concept showing Objective-C
code running on the 360 we are off to the races.  Any help is appreciated just
not sure if all the pieces/parts exist and/or what I am missing.  So... is this
feasible?  If so... how do I get LLVM to output C code?
>>>>>> 
>>>>>> Thanks,
>>>>>> Kevin
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Kevin,

On Thu, May 27, 2010 at 11:49 AM, Kevin Wooten <kdubb at me.com> wrote:
> To ease our development process, actually hide it completely, we are
planning on leveraging the open nature of Clang and LLVM to create a driver for
these platforms that first rewrites ObjC to C and then calls the platform's
C compiler passing through all command line options. This way the ObjC stage
should be fairly transparent.
>
Folks here at llvmdev usually focus on LLVM optimizer and LLVM code
generator which you're likely to bypass completely if you use
platform's C compiler. It seems cfe-dev may be appropriate list to get
answers for your questions regarding LLVM frontend, known as Clang.

-
Devang

My apologies... the original question was directed here because I didn't
know about the Clang rewriter.  Now that the direction has sharply tacked toward
using that method I will direct any further questions/discussions to that list.

Thanks

On May 27, 2010, at 1:47 PM, Devang Patel wrote:
> Kevin,
> 
> On Thu, May 27, 2010 at 11:49 AM, Kevin Wooten <kdubb at me.com>
wrote:
> 
>> To ease our development process, actually hide it completely, we are
planning on leveraging the open nature of Clang and LLVM to create a driver for
these platforms that first rewrites ObjC to C and then calls the platform's
C compiler passing through all command line options. This way the ObjC stage
should be fairly transparent.
>> 
> 
> Folks here at llvmdev usually focus on LLVM optimizer and LLVM code
> generator which you're likely to bypass completely if you use
> platform's C compiler. It seems cfe-dev may be appropriate list to get
> answers for your questions regarding LLVM frontend, known as Clang.
> 
> -
> Devang