llvm dev - Mar 2009 - [LLVMdev] MachO and ELF Writers/MachineCodeEmittersarehard-codedinto LLVMTargetMachine

>> Sorry, I disagree actually the MachineCodeEmitter or the
>> 'MachineCodeWritter' does not do any file handling at all. Do
look at the
>> code for the MachineCodeWritter and you will see it only writes to
memory
>> and if it reaches the end of the allotted memory I believe higher
ordered
>> logic reallocates a larget buffer and starts again from scratch. This 
>> could
>> be avoided if they generated fixus for absolute memory references 
>> refering
>> within the outputted code. Then a alloc function could be called before
>> outputting say a 4 byte int and could realloc and copy code and when 
>> finally
>> written the fixups could be applied.
>
> IIRC the memory allocation is done in the MachineCodeEmitter, not the
> higher level (see startFunction and finishFunction). The current
> implementation has startFunction allocate some (arbitrary) reserve
> size in the output vector, and if we the emitter runs out of space,
> finishFunction returns a failure, causing the whole process to occur
> again. This is icky.
As I said the MachineCodeEmitter works exclusively with the JITEmitter class 
which inherits from the MachineCodeEmitter class and thats where the 
allocation is going on.
> It would be far better if the underlying buffer would grow
> automatically (with an allocation method in the base class, as you
> suggested), as code is emitted to it.
I think we leave the JITEmitter and MachineCodeEmitter alone as they work 
and its a pritty difficult thing to replace them without causing any 
regressions.
>> 'ObjectCodeEmitter' looks like the right description to
parallel the
>> MachineCodeEmitter. Its emitting object code to a data stream (which
>> is an object file section) and not direct to a file.
>
> I can live with that. Before you implement anything, can we try and
> define the responsibilities of the various classes?
Yes.
> We have MachineCodeEmitter, which is responsible for actually emitting
> bytes into a buffer for a function. Should it have methods for
> emitting instructions/operands, or should it only work at the byte,
> dword, etc. level?
It works at primatine byte, word, dword level only. the *CodeEmitter classes 
like the X86CodeEmitter class are responsible for generating machine code 
and use the MachineCodeEmitter class to do this.
> ObjectCodeEmitter,  is responsible for emission of object 'files'
into
> a memory buffer. This includes handling of all object headers,
> management of sections/segments, symbol and string tables and
It writes to sections in ELFWriter::ELFSection.SectionData which are 
std::vector<unsigned char>.
> relocations. The ObjectCodeEmitter should delegate all actual 'data
> emission' to the MachineCodeEmitter.
No it will have to parallel the MachineCodeEmitter as a generic alternative.
> ObjectCodeEmitter is a MachineFunctionPass. It does 'object wide'
> setup in doInitialization and finalizes the object in doFinalize(?).
> Each MachineFunction is emitted through the runOnFunction method,
> which passes the MachineFunction to the MachineCodeEmitter. The
> MachineCodeEmitter calls back to the ObjectCodeEmitter in order to
> look up sections/segments, add globals to an unresolved globals list
> etc.
>
No look at the ELFWriter which inherits from MachineFunctionPass and 
ELFCodeEmitter which inherits from the MachineCodeEmitter.

We need to override the behaviour of the MachineCodeEmitter creating the 
ObjectCodeEmitter which writes to the appropriate section in 
ELFWriter::ELFSection.SectionData.
> I'm not too happy about the broken encapsulation here. I'd prefer
to
> find a better way to model this.
Where ? How ?

Aaron

Aaron, I mailed in the same mail twice (by mistake), you answered both
copies. Differently!

In any case, I've re-read what exists. I'm dumping what I understand
here, so that we can discuss in detail. I'm using MachO as the example
object format, as the ELF code is totally broken and outdated. Lets
use the following as the basis for our discussion?

There are 3 classes which participate in object file emission:
1. MachOWriter
    - a MachineFunctionPass, with a donothing runOnMachineFunction.

doInitialization and doFinalization are used to emit the object file
header and finalize the various object file segments, respectively.

The MachOWriter is responsible for creation of MachOCodeEmitter, via
it's getMachineCodeEmitter function.

2. MachOCodeEmitter
   - a MachineCodeEmitter. Friend class of MachOWriter (friend class
== broken encapsulation!?)

startFunction allocates storage in the text section for the current function.

finishFunction emits constant-pools, jumptables; transforms
relocations adding globals to the MachOWriter's PendingGlobals list,
and all relocations to the parent section's relocation list; adds a
symbol for the function to the MachOWriter's SymbolTable.

[In general, all the operations in finishFunction actually modify the
data of the MachOWriter. Shouldn't these be pushed into the
MachOWriter? ]


3. X86CodeEmitter
   - a MachineFunctionPass, NOT a MachineCodeEmitter (Could the naming
change perhaps?)

This class receives (during construction) a reference to a
MachineCodeEmitter (e.g. MachOCodeEmitter, which in turn stores a
reference to a MachOWritter).

The runOnMachineFunction for the X86CodeEmitter does:
- call MachOCodeEmitter::startFunction
- for each basicblock in function:
   - call MachOCodeEmitter::StartMachineBasicBlock
   - for each instruction in basicblock:
       - emit instruction, using MachineCodeEmitter::emit* functions
- call MachOCodeEmitter::finishFunction

[This runOnMachineFunction could definitely be generalized, i.e.
implemented in a base class ('EmitterMachineFunctionPass' or a better
name). This base class would then have (abstract) emitInstruction,
emitOperand, etc... methods. It should also integrate with the
*GenCodeEmitter emitted by tblgen so that you get automatic code
emission. When implementing a new target, one would simply need to
inherit the baseclass, and override the functions necessary to tweak
output.]

> Aaron, I mailed in the same mail twice (by mistake), you answered both
> copies. Differently!
>
> In any case, I've re-read what exists. I'm dumping what I
understand
> here, so that we can discuss in detail. I'm using MachO as the example
> object format, as the ELF code is totally broken and outdated. Lets
> use the following as the basis for our discussion?
I've never looked at the MachO code as I do not have such a platform nor do 
I know the file format.

Could we concentrate on the ELF backend, please.

> 3. X86CodeEmitter
>   - a MachineFunctionPass, NOT a MachineCodeEmitter (Could the naming
> change perhaps?)
Yes, it uses a MachineCodeEmitter in its internals (MCE) and as a 
constructor argument.
> This class receives (during construction) a reference to a
> MachineCodeEmitter (e.g. MachOCodeEmitter, which in turn stores a
> reference to a MachOWritter).
>
> The runOnMachineFunction for the X86CodeEmitter does:
> - call MachOCodeEmitter::startFunction
> - for each basicblock in function:
>   - call MachOCodeEmitter::StartMachineBasicBlock
>   - for each instruction in basicblock:
>       - emit instruction, using MachineCodeEmitter::emit* functions
> - call MachOCodeEmitter::finishFunction
>
> [This runOnMachineFunction could definitely be generalized, i.e.
> implemented in a base class ('EmitterMachineFunctionPass' or a
better
> name). This base class would then have (abstract) emitInstruction,
> emitOperand, etc... methods. It should also integrate with the
> *GenCodeEmitter emitted by tblgen so that you get automatic code
> emission. When implementing a new target, one would simply need to
> inherit the baseclass, and override the functions necessary to tweak
> output.]
runOnMachineFunction is a standard LLVM message we cannot play around with 
it.

Please read LLVM code more and get a general overview of the standard 
interfaces.

Aaron

> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev