Peter Collingbourne
2014-Jul-08 18:43 UTC
[LLVMdev] Proposal: support object file-based on-disk module format
On Tue, Jul 08, 2014 at 07:21:00PM +0100, Dan Liew wrote:> Hi Peter, > > This sounds sensible to me. > > There is one thing that does concern me though. IIRC when you create > object files with additional sections the GNU ld linker (possibly > others too) will concatenate sections it doesn't recognise into the > final executable. > There is actually a hacky tool called whole-program-llvm [1] which > actually uses this to get a list of paths to LLVM bitcode files that > make up the final executable. > > If I've understood your proposal correctly then when compiling and > using the GNU ld linker you would end up with all the bitcode files > embedded in the final executable. Is this intentional? > > [1] https://github.com/travitch/whole-program-llvmIf the linker never sees the intermediate object files, this will not happen. This is the case under the current proposal. However, if we codegen into the object files, we might want to make those object files visible to the linker. In which case, the compiler can use an object-format-specific exclude flag [1] to exclude those sections from the executable or DSO. Thanks, -- Peter [1] https://sourceware.org/binutils/docs/as/Section.html
David Chisnall
2014-Jul-09 07:35 UTC
[LLVMdev] Proposal: support object file-based on-disk module format
On 8 Jul 2014, at 19:43, Peter Collingbourne <peter at pcc.me.uk> wrote:> If the linker never sees the intermediate object files, this will not > happen. This is the case under the current proposal. However, if we codegen > into the object files, we might want to make those object files visible to > the linker. In which case, the compiler can use an object-format-specific > exclude flag [1] to exclude those sections from the executable or DSO.We (current hat: FreeBSD) would like to be able to leave the LLVM IR in programs and shared libraries so that packaging tools can run microarchitecure-specific optimisations on the result, either offline or at install time (and a few other things, including applying software diversity techniques and so on). David
Dan Liew
2014-Jul-09 09:45 UTC
[LLVMdev] Proposal: support object file-based on-disk module format
> If the linker never sees the intermediate object files, this will not > happen. This is the case under the current proposal.Ah I see. I completely misread your original post, for some reason I thought you were proposing object files that had LLVM IR and the corresponding codegen together. Sorry for the noise.> However, if we codegen > into the object files, we might want to make those object files visible to > the linker. In which case, the compiler can use an object-format-specific > exclude flag [1] to exclude those sections from the executable or DSO.> [1] https://sourceware.org/binutils/docs/as/Section.htmlCool I did not know about those flags. Thanks, Dan.
Peter Collingbourne
2014-Jul-09 18:06 UTC
[LLVMdev] Proposal: support object file-based on-disk module format
On Wed, Jul 09, 2014 at 08:35:18AM +0100, David Chisnall wrote:> On 8 Jul 2014, at 19:43, Peter Collingbourne <peter at pcc.me.uk> wrote: > > > If the linker never sees the intermediate object files, this will not > > happen. This is the case under the current proposal. However, if we codegen > > into the object files, we might want to make those object files visible to > > the linker. In which case, the compiler can use an object-format-specific > > exclude flag [1] to exclude those sections from the executable or DSO. > > We (current hat: FreeBSD) would like to be able to leave the LLVM IR in programs and shared libraries so that packaging tools can run microarchitecure-specific optimisations on the result, either offline or at install time (and a few other things, including applying software diversity techniques and so on).That's an interesting use case, but I think it is to some extent orthogonal to changes to the intermediate object format. It should be possible to teach the LTO plugin to emit the combined bitcode in a (non-excluded) .llvmbc section into the object file that the linker sees. Thanks, -- Peter