[LLVMdev] MemoryBuffer/raw_ostream hybrid for linker?

Nick Kledzik
Fri, 04 May 2012 01:10:08 -0000

Existing llvm code tends to use raw_ostream for writing files.  But raw_ostream
is not a good match for a linker for a couple of reasons:

1) When the linker creates an executable, the file needs the 'x' bit
set.  Currently raw_fd_ostream has no way to set that.

2) The Unix conformance suite actually has some test cases where the linker is
run and the output file does exists but is not writable, or is not writable but
is in a writable directory, or with funky umask values.   raw_fd_ostream
interface has no way to match those semantics.

3) On darwin we have found the linker performs better if it opens the output
file, truncates it to the output size, then mmaps in the file, then writes
directly into that memory buffer.  This avoids the memory copy from the private
buffer to the OS file system buffer in the write() syscall.

4) In the model we are using for lld, a streaming output interface is not
optimal.   Currently, lld copies chunks of code from the (read-only) input
files, to a temporary buffer, then applies any fixups (relocations), then
streams out that temporary buffer.  If instead we had a big output buffer, the
linker could copy the code chunks directly to the output buffer and apply the
fixups there, avoiding an extra copy.

Is there an existing solution for these issues in llvm I've overlooked?
I've searched the bug database and did not find any similar requests.

Should I propose a new llvm/Support/ class?

-Nick