Hi all,
We plan to add some auto-linking support for Mach-O, and need a scheme for
encoding this information in the LLVM IR. We would like the same scheme to
be able to support Microsoft's #pragma comment(lib,...) and #pragma
comment(library, ...) features eventually.
The current proposal is as follows:
--
#1. Extend module-level metadata flags (llvm.module.flags) to support two
new behaviors:
llvm::Module::Append - The value must be a list. Module flags with
equivalent unique IDs and this behavior will be appended in the order that
they are linked.
llvm::Module::AppendUnique - The value must be a list. As with
llvm::Module::Append, module flags with equivalent unique IDs are appended
in the order that they are linked. However, identical MDNodes will only
appear once in the list (at the earliest seen position).
#2. Define a new "known" module level flag metadata "Linker
Options" which
is a list of lists of metadata strings corresponding to linker options.
This metadata flag will use the llvm::Module::AppendUnique option.
The options are expected to be linker specific (thus target specific), we
make no attempt to encode the intent of the options at the IR level. The
frontend is responsible for selecting appropriate options based on the
target.
The module level linker will only unique option lists, any diagnosis of
otherwise redundant or conflicting options is expected to be handled by the
linker.
Example metadata for a module which is expected to link against libz and
the Cocoa framework::
!0 = metadata !{ i32 6, "Linker Options",
metadata !{
!metadata { metadata !"-lz" },
!metadata { metadata !"-framework", metadata
!"Cocoa" } } }
!llvm.module.flags = !{ !0 }
--
We have debated whether or not there is value in adopting a more strict
schema for the metadata (i.e., declare intent more explicitly by encoding
things like static library vs dynamic library vs framework library instead
of using target-specific linker options). However, as we have no
expectation that the compiler will want to inspect this data, it seems like
this adds complexity (and reduces flexibility) for no benefit. It does,
however, have the downside that the frontend needs to participate (and have
target linker knowledge) in order to use the appropriate options.
Other points of discussion:
#1. On Mach-O, the linker will expect and enforce that each list of options
corresponds to a single library. This will not be enforced (nor is it
enforceable) at the IR level, and other targets wouldn't have this
restriction (Visual Studio supports inclusion of some arbitrary flags).
#2. On Microsoft, #pragma comment(linker, ...) will map naturally to this
format. How #pragma comment(lib, ...) gets handled will probably depend on
the details of how this is encoded in the COFF object files, which I am not
yet familiar with.
#3. We make no attempt to encode ordering information amongst the options,
which limits the utility for linking against static libraries. The current
expectation is that this feature be used for system libraries where the
order of the options is not important. A schema that would allow encoding
dependencies amongst libraries to be auto-linked would be substantially
more complicated and is outside the scope of this proposal.
Please let me know if you have an opinion on this works, particularly if
you want to chime in on how this might interact with ELF or COFF.
- Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130115/197f9e07/attachment.html>
On Tue, Jan 15, 2013 at 1:17 PM, Daniel Dunbar <daniel at zuster.org> wrote:> Hi all, > > We plan to add some auto-linking support for Mach-O, and need a scheme for > encoding this information in the LLVM IR. We would like the same scheme to > be able to support Microsoft's #pragma comment(lib,...) and #pragma > comment(library, ...) features eventually. > > The current proposal is as follows: > > -- > > #1. Extend module-level metadata flags (llvm.module.flags) to support two > new behaviors: > > llvm::Module::Append - The value must be a list. Module flags with > equivalent unique IDs and this behavior will be appended in the order that > they are linked. > > llvm::Module::AppendUnique - The value must be a list. As with > llvm::Module::Append, module flags with equivalent unique IDs are appended > in the order that they are linked. However, identical MDNodes will only > appear once in the list (at the earliest seen position). > > #2. Define a new "known" module level flag metadata "Linker Options" which > is a list of lists of metadata strings corresponding to linker options. This > metadata flag will use the llvm::Module::AppendUnique option. > > The options are expected to be linker specific (thus target specific), we > make no attempt to encode the intent of the options at the IR level. The > frontend is responsible for selecting appropriate options based on the > target. > > The module level linker will only unique option lists, any diagnosis of > otherwise redundant or conflicting options is expected to be handled by the > linker. > > Example metadata for a module which is expected to link against libz and the > Cocoa framework:: > > !0 = metadata !{ i32 6, "Linker Options", > metadata !{ > !metadata { metadata !"-lz" }, > !metadata { metadata !"-framework", metadata !"Cocoa" } } } > !llvm.module.flags = !{ !0 } > > -- > > We have debated whether or not there is value in adopting a more strict > schema for the metadata (i.e., declare intent more explicitly by encoding > things like static library vs dynamic library vs framework library instead > of using target-specific linker options). However, as we have no expectation > that the compiler will want to inspect this data, it seems like this adds > complexity (and reduces flexibility) for no benefit. It does, however, have > the downside that the frontend needs to participate (and have target linker > knowledge) in order to use the appropriate options. > > Other points of discussion: > > #1. On Mach-O, the linker will expect and enforce that each list of options > corresponds to a single library. This will not be enforced (nor is it > enforceable) at the IR level, and other targets wouldn't have this > restriction (Visual Studio supports inclusion of some arbitrary flags). > > #2. On Microsoft, #pragma comment(linker, ...) will map naturally to this > format. How #pragma comment(lib, ...) gets handled will probably depend on > the details of how this is encoded in the COFF object files, which I am not > yet familiar with.On COFF #pragma comment(lib, ...) just gets turned into a linker flag in the .drectve section.> > #3. We make no attempt to encode ordering information amongst the options, > which limits the utility for linking against static libraries. The current > expectation is that this feature be used for system libraries where the > order of the options is not important. A schema that would allow encoding > dependencies amongst libraries to be auto-linked would be substantially more > complicated and is outside the scope of this proposal.What do you mean by amongst the options? #pragma comment(lib, ...) requires that the options are added in the order they occur in the source file. This should be trivial to implement in the metadata.> > Please let me know if you have an opinion on this works, particularly if you > want to chime in on how this might interact with ELF or COFF. > > - DanielI don't know of any ELF implementation that currently supports this in general, but adding it should't be too hard. Go does automatic linking, but it now has its own object file format that handles it. gccgo for ELF seems to produce .go_export sections and <package>..import symbols which are both specific to Go. - Michael Spencer
Hi Michael, Thanks for the feedback. On Tue, Jan 15, 2013 at 3:18 PM, Michael Spencer <bigcheesegs at gmail.com>wrote:> On Tue, Jan 15, 2013 at 1:17 PM, Daniel Dunbar <daniel at zuster.org> wrote: > > Hi all, > > > > We plan to add some auto-linking support for Mach-O, and need a scheme > for > > encoding this information in the LLVM IR. We would like the same scheme > to > > be able to support Microsoft's #pragma comment(lib,...) and #pragma > > comment(library, ...) features eventually. > > > > The current proposal is as follows: > > > > -- > > > > #1. Extend module-level metadata flags (llvm.module.flags) to support two > > new behaviors: > > > > llvm::Module::Append - The value must be a list. Module flags with > > equivalent unique IDs and this behavior will be appended in the order > that > > they are linked. > > > > llvm::Module::AppendUnique - The value must be a list. As with > > llvm::Module::Append, module flags with equivalent unique IDs are > appended > > in the order that they are linked. However, identical MDNodes will only > > appear once in the list (at the earliest seen position). > > > > #2. Define a new "known" module level flag metadata "Linker Options" > which > > is a list of lists of metadata strings corresponding to linker options. > This > > metadata flag will use the llvm::Module::AppendUnique option. > > > > The options are expected to be linker specific (thus target specific), we > > make no attempt to encode the intent of the options at the IR level. The > > frontend is responsible for selecting appropriate options based on the > > target. > > > > The module level linker will only unique option lists, any diagnosis of > > otherwise redundant or conflicting options is expected to be handled by > the > > linker. > > > > Example metadata for a module which is expected to link against libz and > the > > Cocoa framework:: > > > > !0 = metadata !{ i32 6, "Linker Options", > > metadata !{ > > !metadata { metadata !"-lz" }, > > !metadata { metadata !"-framework", metadata !"Cocoa" } } } > > !llvm.module.flags = !{ !0 } > > > > -- > > > > We have debated whether or not there is value in adopting a more strict > > schema for the metadata (i.e., declare intent more explicitly by encoding > > things like static library vs dynamic library vs framework library > instead > > of using target-specific linker options). However, as we have no > expectation > > that the compiler will want to inspect this data, it seems like this adds > > complexity (and reduces flexibility) for no benefit. It does, however, > have > > the downside that the frontend needs to participate (and have target > linker > > knowledge) in order to use the appropriate options. > > > > Other points of discussion: > > > > #1. On Mach-O, the linker will expect and enforce that each list of > options > > corresponds to a single library. This will not be enforced (nor is it > > enforceable) at the IR level, and other targets wouldn't have this > > restriction (Visual Studio supports inclusion of some arbitrary flags). > > > > #2. On Microsoft, #pragma comment(linker, ...) will map naturally to this > > format. How #pragma comment(lib, ...) gets handled will probably depend > on > > the details of how this is encoded in the COFF object files, which I am > not > > yet familiar with. > > On COFF #pragma comment(lib, ...) just gets turned into a linker flag > in the .drectve section. >Ok. Can you elaborate a bit more on what that looks like? Does it end up in the same format as #pragma comment(linker, ...), or does the COFF writer need to be able to discriminate between them?> #3. We make no attempt to encode ordering information amongst the options, > > which limits the utility for linking against static libraries. The > current > > expectation is that this feature be used for system libraries where the > > order of the options is not important. A schema that would allow encoding > > dependencies amongst libraries to be auto-linked would be substantially > more > > complicated and is outside the scope of this proposal. > > What do you mean by amongst the options? #pragma comment(lib, ...) > requires that the options are added in the order they occur in the > source file. This should be trivial to implement in the metadata. >Ah, I glossed over this. Naturally we will retain things in the order they are present in the IR, what I meant was there is no attempt to define a more complex dependency scheme (i.e. encode dependencies among individual options so that linking essentially would topologically sort them).> > > > > Please let me know if you have an opinion on this works, particularly if > you > > want to chime in on how this might interact with ELF or COFF. > > > > - Daniel > > I don't know of any ELF implementation that currently supports this in > general, but adding it should't be too hard. >Right. Most of the work is in the linker side where it may involve additional policy decisions. On Windows, for example, the linker handles options that come from .obj files differently than other ones, its not just a matter of adding them in to the user specified ones. On Mach-O, the linker will handle such options slightly differently than command line options and only use them as necessary (to resolve missing symbols), but not treat them as required to be linked. - Daniel> Go does automatic linking, but it now has its own object file format > that handles it. gccgo for ELF seems to produce .go_export sections > and <package>..import symbols which are both specific to Go. > > - Michael Spencer >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130116/8cddf658/attachment.html>
Hi Daniel,
Nice to meet you.
My understanding of the Microsoft #pragma comment(lib, ...) semantics is that
each specified library will be converted into a directive that starts with
"/DEFAULTLIB" in the COFF .drectve section. To demonstrate, the
following patch
produces directives that work with Visual Studio 2010 using the now-deprecated
dependent library feature (commits r168779 and r168694).
/* beginning of patch */
Index: /home/ygao/LLVM/llvm/lib/Target/X86/X86AsmPrinter.cpp
==================================================================---
/home/ygao/LLVM/llvm/lib/Target/X86/X86AsmPrinter.cpp (revision 165914)
+++ /home/ygao/LLVM/llvm/lib/Target/X86/X86AsmPrinter.cpp (working copy)
@@ -641,6 +641,19 @@
const TargetLoweringObjectFileCOFF &TLOFCOFF
static_cast<const
TargetLoweringObjectFileCOFF&>(getObjFileLowering());
+ // Output linker support code for #pragma comment(lib, ...) on Windows
+ if (M.lib_size() > 0 && Subtarget->isTargetWindows()) {
+ OutStreamer.SwitchSection(TLOFCOFF.getDrectveSection());
+ SmallString<128> name;
+ for (Module::lib_iterator I = M.lib_begin(), E = M.lib_end();
+ I != E; ++I) {
+ name = " /DEFAULTLIB:\"";
+ name += *I;
+ name += "\"";
+ OutStreamer.EmitBytes(name, 0);
+ }
+ }
+
for (Module::const_iterator I = M.begin(), E = M.end(); I != E; ++I)
if (I->hasDLLExportLinkage())
DLLExportedFns.push_back(Mang->getSymbol(I));
/* end of patch */
I am currently looking for advice on what is the correct approach to
re-implement #pragma comment(lib, ...) with Clang/LLVM.
A mingw linker (at least the one that I played with) just ignores this directive
with no warnings or errors.
One can either add another module flag for "Named Libraries" in
addition to
"Linker Options", which might be easier to implement. Or one can
provide a
target-specific hook to the frontend, which converts #pragma comment(lib,
"blah.lib") to something like this:
!0 = medata !{ i32 6, "Linker Options",
metadata !{
!metadata { metadata !"/DEFAULTLIB:blah.lib" }
}
}
!llvm.module.flags = !{ !0 }
Your example is maybe more complicated where you need to convert libz to
"-lz".
Regarding the ordering information of the libraries, I think for MSVC the
libraries are supposed to follow lexical order in the same source file. I am not
sure if there is any restriction across different source files. Here is a link
to the MSDN doc:
http://msdn.microsoft.com/en-us/library/7f0aews7%28v=vs.80%29.aspx
It is probably not too hard to add a directive section to ELF similar to COFF.
At the 2012 LLVM Dev Meeting, there was a presentation on adding Modules to C,
which presented a way to use ObjC-style .map files to specify libraries, which
may be relevant to this auto-linking feature.
I look forward to more discussions,
- Gao.
Hi Gao, Thanks for the extra information... On Wed, Jan 16, 2013 at 1:14 PM, Yunzhong Gao <Yunzhong.Gao at am.sony.com>wrote:> Hi Daniel, > > Nice to meet you. > > My understanding of the Microsoft #pragma comment(lib, ...) semantics is > that > each specified library will be converted into a directive that starts with > "/DEFAULTLIB" in the COFF .drectve section. To demonstrate, the following > patch > produces directives that work with Visual Studio 2010 using the > now-deprecated > dependent library feature (commits r168779 and r168694). >Is that feature now-deprecated? The documentation here: http://msdn.microsoft.com/en-us/library/7f0aews7.aspx only talks about the "exestr" comment type being deprecated. I see that the /defaultlib flag itself to the linker is deprecated, but it isn't clear to me that that means the dependent library feature is also deprecated. /* beginning of patch */> > Index: /home/ygao/LLVM/llvm/lib/Target/X86/X86AsmPrinter.cpp > ==================================================================> --- /home/ygao/LLVM/llvm/lib/Target/X86/X86AsmPrinter.cpp (revision > 165914) > +++ /home/ygao/LLVM/llvm/lib/Target/X86/X86AsmPrinter.cpp (working > copy) > @@ -641,6 +641,19 @@ > const TargetLoweringObjectFileCOFF &TLOFCOFF > static_cast<const > TargetLoweringObjectFileCOFF&>(getObjFileLowering()); > > + // Output linker support code for #pragma comment(lib, ...) on Windows > + if (M.lib_size() > 0 && Subtarget->isTargetWindows()) { > + OutStreamer.SwitchSection(TLOFCOFF.getDrectveSection()); > + SmallString<128> name; > + for (Module::lib_iterator I = M.lib_begin(), E = M.lib_end(); > + I != E; ++I) { > + name = " /DEFAULTLIB:\""; > + name += *I; > + name += "\""; > + OutStreamer.EmitBytes(name, 0); > + } > + } > + > for (Module::const_iterator I = M.begin(), E = M.end(); I != E; ++I) > if (I->hasDLLExportLinkage()) > DLLExportedFns.push_back(Mang->getSymbol(I)); > > /* end of patch */ > > I am currently looking for advice on what is the correct approach to > re-implement #pragma comment(lib, ...) with Clang/LLVM. > > A mingw linker (at least the one that I played with) just ignores this > directive > with no warnings or errors. > > One can either add another module flag for "Named Libraries" in addition to > "Linker Options", which might be easier to implement. Or one can provide a > target-specific hook to the frontend, which converts #pragma comment(lib, > "blah.lib") to something like this: >I would vote that we have a target hook that lets us emit the code into the "Linker Options" metadata, which would also be our handling of #pragma comment(linker, ...)> > !0 = medata !{ i32 6, "Linker Options", > metadata !{ > !metadata { metadata !"/DEFAULTLIB:blah.lib" } > } > } > !llvm.module.flags = !{ !0 } > > Your example is maybe more complicated where you need to convert libz to > "-lz". > > Regarding the ordering information of the libraries, I think for MSVC the > libraries are supposed to follow lexical order in the same source file. I > am not > sure if there is any restriction across different source files. Here is a > link > to the MSDN doc: > http://msdn.microsoft.com/en-us/library/7f0aews7%28v=vs.80%29.aspxSee previous reply to Michael Spencer, but yes, we will preserve order.> It is probably not too hard to add a directive section to ELF similar to > COFF. > > At the 2012 LLVM Dev Meeting, there was a presentation on adding Modules > to C, > which presented a way to use ObjC-style .map files to specify libraries, > which > may be relevant to this auto-linking feature. >Yes, the motivation for adding the feature is to support modules (actually, current trunk sources are already emitting some kind of metadata to this effect, although it needs to be updated). - Daniel> > I look forward to more discussions, > > - Gao. > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130116/0047134b/attachment.html>