llvm dev - Sep 2009 - [LLVMdev] Global register variables/custom calling conventions

Anton Korobeynikov wrote:
> Ok, what's left from QEMU then? :)
The hardware emulation (interrupts, condition flags, register file etc) 
and execution framework (block selection and execution) from qemu are 
still used - translating the ARM to the native architecture is only part 
of the story :)
> 
>> generating reasonable code - this approach keeps it in place while we
do
>> extra, possibly more expensive work out of sight. It might not be a
pretty
>> idea, but LLVM does generate some very tight code :) It's an
experiment -
>> humour me...
> Well, but I still don't get the reason why you need to pin (some)
> internal QEMU state variables to fixed registers?
> 
TCG seperates the guest (ARM) code into blocks - my front end translates 
these to LLVM IR for LLVM to translate to x86.  The assumption is that 
LLVM will produce a better translation than TCG*. At some future point 
the TCG-generated native block is replaced by the LLVM's, and as such it 
needs to hand control back to qemu in a state that it would expect from 
TCG. Essentially the idea is to take the same input and produce the same 
output as the original TCG block, but munge things in the middle to 
(hopefully) be more efficient using LLVM's local optimisations.

All up, the the LLVM generated code needs to conform to what qemu is 
expecting from TCG in terms of emulated register state, and, as it's 
pinning values in host registers, some parts of the host register state. 
The pinned register in this case holds a pointer to the emulated cpu's 
state structure as it's frequently accessed. Clobbering this would break 
things such as direct block chaining (which avoids the need to jump out 
to the dispatch loop to find the next block to execute) and possibly 
other parts of the execution framework.

Hope it makes some sense :)

Cheers,

Andrew

* At this point the frontend only supports a few ARM instructions, and 
it avoids generating IR that could lead to positional dependence. The 
general approach has the nice property that if an instruction that isn't 
implemented is encountered, the block simply isn't translated by LLVM 
and the TCG version lives on.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20090923/f831a415/attachment.sig>

Hi Andrew,

On Wed, Sep 23, 2009 at 7:26 AM, Andrew Jeffery <andrew at aj.id.au>
wrote:
> TCG seperates the guest (ARM) code into blocks - my front end translates
> these to LLVM IR for LLVM to translate to x86.  The assumption is that LLVM
> will produce a better translation than TCG*. At some future point the
> TCG-generated native block is replaced by the LLVM's, and as such it
needs
> to hand control back to qemu in a state that it would expect from TCG.
> Essentially the idea is to take the same input and produce the same output
> as the original TCG block, but munge things in the middle to (hopefully) be
> more efficient using LLVM's local optimisations.
I'm curious, are you translating directly from ARM to LLVM IR or from
TCG IR to LLVM IR?

llvm-qemu just put the pinned variables into memory (and lived with
the performance penalty), but I agree it would be much nicer to have a
custom calling convention in order to avoid this.

Cheers,

Tilmann

On 25/09/09 18:05, Tilmann Scheller wrote:
> Hi Andrew,
>
> On Wed, Sep 23, 2009 at 7:26 AM, Andrew Jeffery<andrew at aj.id.au> 
wrote:
>> TCG seperates the guest (ARM) code into blocks - my front end
translates
>> these to LLVM IR for LLVM to translate to x86.  The assumption is that
LLVM
>> will produce a better translation than TCG*. At some future point the
>> TCG-generated native block is replaced by the LLVM's, and as such
it needs
>> to hand control back to qemu in a state that it would expect from TCG.
>> Essentially the idea is to take the same input and produce the same
output
>> as the original TCG block, but munge things in the middle to
(hopefully) be
>> more efficient using LLVM's local optimisations.
> I'm curious, are you translating directly from ARM to LLVM IR or from
> TCG IR to LLVM IR?
>
> llvm-qemu just put the pinned variables into memory (and lived with
> the performance penalty), but I agree it would be much nicer to have a
> custom calling convention in order to avoid this.
>

I'm translating straight from ARM to LLVM IR, avoiding TCG IR. I've 
taken the approach of implementing each basic block determined by TCG as 
a function in LLVM, which takes a pointer to the ARM CPU state struct as 
a parameter and returns the pointer to the struct. Currently I'm using 
LLVM's JIT to generate the target (x86_64) instructions, and the plan is 
to copy the instructions it generates into the translated code buffer. 
As mentioned in the previous email, the front-end has been designed to 
avoid location specific code and as such should be suitable for copying 
around in memory.

At the moment I'm not explicitly specifying a calling convention for the
functions, but LLVM seems to consistently put the ARM CPU state struct 
pointer (parameter) into %rdi. As a hack-around (since posting the 
original message to the list) I'm injecting a couple of MOV instructions 
to move the pointer from %r14 (AREG0 - TCG's pointer to the CPU state 
struct on x86_64) to %rdi before and back (from %rdi) to %r14 after the 
copied function block. Implementing a custom calling convention would 
avoid the injected MOVs, saving two instructions per block I guess. This 
isn't a huge win, and as such isn't a big priority, however it'd be
a
nice thing to have... It'd make things slightly less hackish anyway.

Cheers,

Andrew