llvm dev - Dec 2011 - [LLVMdev] Proposal for -filetype=obj full big endian support

Jim,

I see routines that are already available to do the endianizing, but the data
and text section contents don't have sizing for the individual elements as
far as I can see so that I can endianize them. That is the part I am trying to
solve, not the bit twiddling algorithm.

Did I miss something?

Jack
________________________________________
From: Jim Grosbach [grosbach at apple.com]
Sent: Tuesday, December 20, 2011 11:58 AM
To: Carter, Jack
Cc: List
Subject: Re: [LLVMdev] Proposal for -filetype=obj full big endian support

Hi Jack,

Everything should be already using the sized Emit* routines directly rather than
outputting individual bytes of larger entities piecemeal. Have you found that
not to be the case?

The Emit* routines handle endianness for you, so you shouldn't have to do
much beyond that. For example, here's the EmitInvValue() implementation that
underlies things and does the endianness transform:

/// EmitIntValue - Special case of EmitValue that avoids the client having to
/// pass in a MCExpr for constant integers.
void MCStreamer::EmitIntValue(uint64_t Value, unsigned Size,
                              unsigned AddrSpace) {
  assert(Size <= 8 && "Invalid size");
  assert((isUIntN(8 * Size, Value) || isIntN(8 * Size, Value)) &&
         "Invalid size");
  char buf[8];
  const bool isLittleEndian = Context.getAsmInfo().isLittleEndian();
  for (unsigned i = 0; i != Size; ++i) {
    unsigned index = isLittleEndian ? i : (Size - i - 1);
    buf[i] = uint8_t(Value >> (index * 8));
  }
  EmitBytes(StringRef(buf, Size), AddrSpace);
}

-Jim

On Dec 19, 2011, at 5:17 PM, Carter, Jack wrote:
> Proposal for ELF text and data big endian support for direct object
generation
>
> Unless I am mistaken, currently big endian support in the MC layer for ELF
> direct object generation is limited to ELF headers and tables like
relocations.
> Text and data section contents are still generated as individual bytes.
> It looks as if the effort was started, but never completed.
>
> The proposal is to extend the MCDataFragment class to include a container
> of offset/size that one can traverse generating each element in the correct
> endianess:
>
>   // Pseudo code
> for (iterator it = begin(), ie = end(); it != ie; ++it) {
>      switch (it.getValueSize()) {
>       default:
>         assert(0 && "Invalid size!");
>       case 1: OW->Write8 (uint8_t (it.getValue())); break;
>       case 2: OW->Write16(uint16_t(it.getValue())); break;
>       case 4: OW->Write32(uint32_t(it.getValue())); break;
>       case 8: OW->Write64(uint64_t(it.getValue())); break;
>       }
>     }
>
> Note: ObjectWriter has a set of endian correcting routines as use above.
>
> The output routines like the snippet above would be in
lib/MC/MCAssembler.cpp
>
> There is an MCInstFragment, but it doesn't seem to be used. All text
and data
> seem to go through MCDataFragment.
>
> The offset/size container would be filled at the
> lib/Taget/<your target>/MCTargetDesc level. In <your
targetMCCODEEmitter.cpp
> for at least text. Some other source file for data.
>
> It would be nice if someone could prove that the support is already there
> and I don't have to do anything to get it, but if not, is the above
> strawman reasonable? If so, I'll implement it and put it up for review.
>
> Cheers,
>
> Jack
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Hi Jack,

I would have expected the Mips backend to be using these routines (or more
likely something on top of them) to emit the contents of the data and text
sections and thus have the bit-twiddling done by these routines. I take it
that's not happening?

Basically, this should already work without any additional infrastructure. If
it's not, something is broken and bypassing the mechanisms they should be
using to output data.

-Jim

On Dec 20, 2011, at 12:21 PM, Carter, Jack wrote:
> Jim,
> 
> I see routines that are already available to do the endianizing, but the
data and text section contents don't have sizing for the individual elements
as far as I can see so that I can endianize them. That is the part I am trying
to solve, not the bit twiddling algorithm.
> 
> Did I miss something?
> 
> Jack
> ________________________________________
> From: Jim Grosbach [grosbach at apple.com]
> Sent: Tuesday, December 20, 2011 11:58 AM
> To: Carter, Jack
> Cc: List
> Subject: Re: [LLVMdev] Proposal for -filetype=obj full big endian support
> 
> Hi Jack,
> 
> Everything should be already using the sized Emit* routines directly rather
than outputting individual bytes of larger entities piecemeal. Have you found
that not to be the case?
> 
> The Emit* routines handle endianness for you, so you shouldn't have to
do much beyond that. For example, here's the EmitInvValue() implementation
that underlies things and does the endianness transform:
> 
> /// EmitIntValue - Special case of EmitValue that avoids the client having
to
> /// pass in a MCExpr for constant integers.
> void MCStreamer::EmitIntValue(uint64_t Value, unsigned Size,
>                              unsigned AddrSpace) {
>  assert(Size <= 8 && "Invalid size");
>  assert((isUIntN(8 * Size, Value) || isIntN(8 * Size, Value)) &&
>         "Invalid size");
>  char buf[8];
>  const bool isLittleEndian = Context.getAsmInfo().isLittleEndian();
>  for (unsigned i = 0; i != Size; ++i) {
>    unsigned index = isLittleEndian ? i : (Size - i - 1);
>    buf[i] = uint8_t(Value >> (index * 8));
>  }
>  EmitBytes(StringRef(buf, Size), AddrSpace);
> }
> 
> -Jim
> 
> On Dec 19, 2011, at 5:17 PM, Carter, Jack wrote:
> 
>> Proposal for ELF text and data big endian support for direct object
generation
>> 
>> Unless I am mistaken, currently big endian support in the MC layer for
ELF
>> direct object generation is limited to ELF headers and tables like
relocations.
>> Text and data section contents are still generated as individual bytes.
>> It looks as if the effort was started, but never completed.
>> 
>> The proposal is to extend the MCDataFragment class to include a
container
>> of offset/size that one can traverse generating each element in the
correct
>> endianess:
>> 
>>  // Pseudo code
>> for (iterator it = begin(), ie = end(); it != ie; ++it) {
>>     switch (it.getValueSize()) {
>>      default:
>>        assert(0 && "Invalid size!");
>>      case 1: OW->Write8 (uint8_t (it.getValue())); break;
>>      case 2: OW->Write16(uint16_t(it.getValue())); break;
>>      case 4: OW->Write32(uint32_t(it.getValue())); break;
>>      case 8: OW->Write64(uint64_t(it.getValue())); break;
>>      }
>>    }
>> 
>> Note: ObjectWriter has a set of endian correcting routines as use
above.
>> 
>> The output routines like the snippet above would be in
lib/MC/MCAssembler.cpp
>> 
>> There is an MCInstFragment, but it doesn't seem to be used. All
text and data
>> seem to go through MCDataFragment.
>> 
>> The offset/size container would be filled at the
>> lib/Taget/<your target>/MCTargetDesc level. In <your
targetMCCODEEmitter.cpp
>> for at least text. Some other source file for data.
>> 
>> It would be nice if someone could prove that the support is already
there
>> and I don't have to do anything to get it, but if not, is the above
>> strawman reasonable? If so, I'll implement it and put it up for
review.
>> 
>> Cheers,
>> 
>> Jack
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

On Dec 20, 2011, at 1:04 PM, Jim Grosbach wrote:
> Hi Jack,
> 
> I would have expected the Mips backend to be using these routines (or more
likely something on top of them) to emit the contents of the data and text
sections and thus have the bit-twiddling done by these routines. I take it
that's not happening?
Clarification: I take it the bit-twiddling isn't happening. I don't know
the mips backend well enough to know which APIs it's using to emit stuff.
Didn't intend to imply otherwise.
> 
> Basically, this should already work without any additional infrastructure.
If it's not, something is broken and bypassing the mechanisms they should be
using to output data.
> 
> -Jim
> 
> On Dec 20, 2011, at 12:21 PM, Carter, Jack wrote:
> 
>> Jim,
>> 
>> I see routines that are already available to do the endianizing, but
the data and text section contents don't have sizing for the individual
elements as far as I can see so that I can endianize them. That is the part I am
trying to solve, not the bit twiddling algorithm.
>> 
>> Did I miss something?
>> 
>> Jack
>> ________________________________________
>> From: Jim Grosbach [grosbach at apple.com]
>> Sent: Tuesday, December 20, 2011 11:58 AM
>> To: Carter, Jack
>> Cc: List
>> Subject: Re: [LLVMdev] Proposal for -filetype=obj full big endian
support
>> 
>> Hi Jack,
>> 
>> Everything should be already using the sized Emit* routines directly
rather than outputting individual bytes of larger entities piecemeal. Have you
found that not to be the case?
>> 
>> The Emit* routines handle endianness for you, so you shouldn't have
to do much beyond that. For example, here's the EmitInvValue()
implementation that underlies things and does the endianness transform:
>> 
>> /// EmitIntValue - Special case of EmitValue that avoids the client
having to
>> /// pass in a MCExpr for constant integers.
>> void MCStreamer::EmitIntValue(uint64_t Value, unsigned Size,
>>                             unsigned AddrSpace) {
>> assert(Size <= 8 && "Invalid size");
>> assert((isUIntN(8 * Size, Value) || isIntN(8 * Size, Value)) &&
>>        "Invalid size");
>> char buf[8];
>> const bool isLittleEndian = Context.getAsmInfo().isLittleEndian();
>> for (unsigned i = 0; i != Size; ++i) {
>>   unsigned index = isLittleEndian ? i : (Size - i - 1);
>>   buf[i] = uint8_t(Value >> (index * 8));
>> }
>> EmitBytes(StringRef(buf, Size), AddrSpace);
>> }
>> 
>> -Jim
>> 
>> On Dec 19, 2011, at 5:17 PM, Carter, Jack wrote:
>> 
>>> Proposal for ELF text and data big endian support for direct object
generation
>>> 
>>> Unless I am mistaken, currently big endian support in the MC layer
for ELF
>>> direct object generation is limited to ELF headers and tables like
relocations.
>>> Text and data section contents are still generated as individual
bytes.
>>> It looks as if the effort was started, but never completed.
>>> 
>>> The proposal is to extend the MCDataFragment class to include a
container
>>> of offset/size that one can traverse generating each element in the
correct
>>> endianess:
>>> 
>>> // Pseudo code
>>> for (iterator it = begin(), ie = end(); it != ie; ++it) {
>>>    switch (it.getValueSize()) {
>>>     default:
>>>       assert(0 && "Invalid size!");
>>>     case 1: OW->Write8 (uint8_t (it.getValue())); break;
>>>     case 2: OW->Write16(uint16_t(it.getValue())); break;
>>>     case 4: OW->Write32(uint32_t(it.getValue())); break;
>>>     case 8: OW->Write64(uint64_t(it.getValue())); break;
>>>     }
>>>   }
>>> 
>>> Note: ObjectWriter has a set of endian correcting routines as use
above.
>>> 
>>> The output routines like the snippet above would be in
lib/MC/MCAssembler.cpp
>>> 
>>> There is an MCInstFragment, but it doesn't seem to be used. All
text and data
>>> seem to go through MCDataFragment.
>>> 
>>> The offset/size container would be filled at the
>>> lib/Taget/<your target>/MCTargetDesc level. In <your
targetMCCODEEmitter.cpp
>>> for at least text. Some other source file for data.
>>> 
>>> It would be nice if someone could prove that the support is already
there
>>> and I don't have to do anything to get it, but if not, is the
above
>>> strawman reasonable? If so, I'll implement it and put it up for
review.
>>> 
>>> Cheers,
>>> 
>>> Jack
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev