Adrian Prantl
2015-Feb-11 21:15 UTC
[LLVMdev] COFF on-disk section alignment Was: Re: [cfe-dev] [PATCH] Wrap clang modules inside Mach-O/ELF/COFF containers
> On Feb 11, 2015, at 12:57 PM, David Majnemer <david.majnemer at gmail.com> wrote: > > > > On Wed, Feb 11, 2015 at 11:39 AM, Adrian Prantl <aprantl at apple.com <mailto:aprantl at apple.com>> wrote: > > On Jan 30, 2015, at 6:04 PM, Adrian Prantl <aprantl at apple.com <mailto:aprantl at apple.com>> wrote: > > The big drawback is that I can no longer influence the order in which the sections are emitted, which turns out to be a problem for COFF. > > Although COFF sections are marked with the correct alignment characteristics, they are not actually emitted aligned on disk. This is hopefully just a bug in our COFF backend. Since the DWARF sections are of arbitrary size and come before the serialized AST, this messes up the alignment for OnDiskHashTable. Until I figure out a solution for this problem, I’ve disabled module debug info ouput for COFF. > > [CC’ing a bunch of people who recently touched the COFF streamer.] > > I’m creating a new section via a global variable like this: > > auto *ASTSym = new llvm::GlobalVariable( > *M, Ty, /*constant*/ true, llvm::GlobalVariable::InternalLinkage, > Data, "__clang_ast"); > ASTSym->setAlignment(8); > ASTSym->setSection("clangast”); > > and llvm-readobj tells me it comes out as: > > Section { > Number: 12 > Name: clangast (63 6C 61 6E 67 61 73 74) > VirtualSize: 0x0 > VirtualAddress: 0x0 > RawDataSize: 148992 > PointerToRawData: 0x4FB <-- 1275 > PointerToRelocations: 0x0 > PointerToLineNumbers: 0x0 > RelocationCount: 0 > LineNumberCount: 0 > Characteristics [ (0x40400040) > IMAGE_SCN_ALIGN_8BYTES (0x400000) > IMAGE_SCN_CNT_INITIALIZED_DATA (0x40) > IMAGE_SCN_MEM_READ (0x40000000) > ] > } > > or in assembly: > > ... > Lsection_debug_loc: > .section .debug_ranges,"r" > Ldebug_range: > .section clangast,"dr" > .align 8 > ___clang_ast: > ... > > Even though the section has the IMAGE_SCN_ALIGN_8BYTES characteristic and the symbol is aligned, the ___clang_ast symbol is still emitted to disk immediately following the previous section and thus ends up at a random unaligned address. This breaks OnDiskHashtable when the object file is mmapped from disk later. > > Am I using it wrong or is this a bug in the coff backend? > > I don't think this is a bug. Here is the relevant text from the COFF specification for the PointerToRawData field: > For executable images, this must be a multiple of FileAlignment from the optional header. > For object files, the value should be aligned on a 4byte boundary for best performance. > > We aren't required to align the section contents in object files.I see. In this particular case, however, it is not even aligned on a 4-byte boundary (0x4FB==1275). Technically I think 4-byte alignment is all that OnDiskHashTable is asking for, so if we could get that right, it would be good enough for my use-case.> > Any ideas how to fix it? > > Nothing would stop us from having a little padding before the start of the section contents.Do you have a pointer where to look for? thanks! adrian -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150211/ccefee71/attachment.html>
David Majnemer
2015-Feb-11 22:24 UTC
[LLVMdev] COFF on-disk section alignment Was: Re: [cfe-dev] [PATCH] Wrap clang modules inside Mach-O/ELF/COFF containers
I went ahead and implemented this in r228879. On Wed, Feb 11, 2015 at 1:15 PM, Adrian Prantl <aprantl at apple.com> wrote:> > On Feb 11, 2015, at 12:57 PM, David Majnemer <david.majnemer at gmail.com> > wrote: > > > > On Wed, Feb 11, 2015 at 11:39 AM, Adrian Prantl <aprantl at apple.com> wrote: > >> >> On Jan 30, 2015, at 6:04 PM, Adrian Prantl <aprantl at apple.com> wrote: >> > The big drawback is that I can no longer influence the order in which >> the sections are emitted, which turns out to be a problem for COFF. >> > Although COFF sections are marked with the correct alignment >> characteristics, they are not actually emitted aligned on disk. This is >> hopefully just a bug in our COFF backend. Since the DWARF sections are of >> arbitrary size and come before the serialized AST, this messes up the >> alignment for OnDiskHashTable. Until I figure out a solution for this >> problem, I’ve disabled module debug info ouput for COFF. >> >> [CC’ing a bunch of people who recently touched the COFF streamer.] >> >> I’m creating a new section via a global variable like this: >> >> auto *ASTSym = new llvm::GlobalVariable( >> *M, Ty, /*constant*/ true, llvm::GlobalVariable::InternalLinkage, >> Data, "__clang_ast"); >> ASTSym->setAlignment(8); >> ASTSym->setSection("clangast”); >> >> and llvm-readobj tells me it comes out as: >> >> Section { >> Number: 12 >> Name: clangast (63 6C 61 6E 67 61 73 74) >> VirtualSize: 0x0 >> VirtualAddress: 0x0 >> RawDataSize: 148992 >> PointerToRawData: 0x4FB <-- 1275 >> PointerToRelocations: 0x0 >> PointerToLineNumbers: 0x0 >> RelocationCount: 0 >> LineNumberCount: 0 >> Characteristics [ (0x40400040) >> IMAGE_SCN_ALIGN_8BYTES (0x400000) >> IMAGE_SCN_CNT_INITIALIZED_DATA (0x40) >> IMAGE_SCN_MEM_READ (0x40000000) >> ] >> } >> >> or in assembly: >> >> ... >> Lsection_debug_loc: >> .section .debug_ranges,"r" >> Ldebug_range: >> .section clangast,"dr" >> .align 8 >> ___clang_ast: >> ... >> >> Even though the section has the IMAGE_SCN_ALIGN_8BYTES characteristic and >> the symbol is aligned, the ___clang_ast symbol is still emitted to disk >> immediately following the previous section and thus ends up at a random >> unaligned address. This breaks OnDiskHashtable when the object file is >> mmapped from disk later. >> >> Am I using it wrong or is this a bug in the coff backend? >> > > I don't think this is a bug. Here is the relevant text from the COFF > specification for the PointerToRawData field: > For executable images, this must be a multiple of FileAlignment from the > optional header. > For object files, the value should be aligned on a 4byte boundary for best > performance. > > We aren't required to align the section contents in object files. > > > I see. In this particular case, however, it is not even aligned on a > 4-byte boundary (0x4FB==1275). Technically I think 4-byte alignment is all > that OnDiskHashTable is asking for, so if we could get that right, it would > be good enough for my use-case. > > > >> Any ideas how to fix it? >> > > Nothing would stop us from having a little padding before the start of the > section contents. > > > Do you have a pointer where to look for? > > thanks! > adrian >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150211/f1eab1f3/attachment.html>