r143524 changed ModuleLinker such that when linking a Module B into a Module A, any internal/private/... functions in B are only linked into A if there is a use of them in A. This is problematic for my usage of module linking, where I'm basically linking a few modules together (some of which include internal functions) and then generating additional code that may use some of the internal functions--so I'd like everything to be linked in at the start, so that all of those functions are available. I don't want to not mark those functions as internal, since if they're not used, I'd like them to be eliminated when I'm done with compilation. With this change, those functions are now no longer available. If there's a different way I should be doing this (i.e. if I'm doing the wrong thing in the above), I'd be interested to hear a suggestion. Alternatively, as a potential fix, I've attached a patch that adds an option to LinkModule() and LinkInModules() that enables/disables lazy linking. Thanks, -matt -------------- next part -------------- A non-text attachment was scrubbed... Name: llvm.patch Type: application/octet-stream Size: 5861 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20111104/b7d1e82e/attachment.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: clang.patch Type: application/octet-stream Size: 588 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20111104/b7d1e82e/attachment-0001.obj>
On Fri, Nov 4, 2011 at 10:46 AM, Matt Pharr <matt.pharr at gmail.com> wrote:> r143524 changed ModuleLinker such that when linking a Module B into a Module A, any internal/private/... functions in B are only linked into A if there is a use of them in A. This is problematic for my usage of module linking, where I'm basically linking a few modules together (some of which include internal functions) and then generating additional code that may use some of the internal functions--so I'd like everything to be linked in at the start, so that all of those functions are available. I don't want to not mark those functions as internal, since if they're not used, I'd like them to be eliminated when I'm done with compilation. With this change, those functions are now no longer available. > > If there's a different way I should be doing this (i.e. if I'm doing the wrong thing in the above), I'd be interested to hear a suggestion.One way to do something like this is to mark the relevant functions external, then use the -internalize pass once everything is linked together. That should be much less sensitive to how exactly linking works. -Eli
On Nov 4, 2011, at 10:55 AM, Eli Friedman wrote:> On Fri, Nov 4, 2011 at 10:46 AM, Matt Pharr <matt.pharr at gmail.com> wrote: >> r143524 changed ModuleLinker such that when linking a Module B into a Module A, any internal/private/... functions in B are only linked into A if there is a use of them in A. This is problematic for my usage of module linking, where I'm basically linking a few modules together (some of which include internal functions) and then generating additional code that may use some of the internal functions--so I'd like everything to be linked in at the start, so that all of those functions are available. I don't want to not mark those functions as internal, since if they're not used, I'd like them to be eliminated when I'm done with compilation. With this change, those functions are now no longer available. >> >> If there's a different way I should be doing this (i.e. if I'm doing the wrong thing in the above), I'd be interested to hear a suggestion. > > One way to do something like this is to mark the relevant functions > external, then use the -internalize pass once everything is linked > together. That should be much less sensitive to how exactly linking > works.Ah, thanks for the pointer to the internalize pass. I ended up implementing the trivial variant of that that takes a given set of function names to mark as internal, rather than a set of functions to mark as external like it does.>From an API generality/flexibility standpoint, I do think that it's a a step backwards to no longer have a Linker method available that will just absolutely link everything in the two given modules together.Thanks, -matt
Rafael Ávila de Espíndola
2011-Nov-13 17:29 UTC
[LLVMdev] Problems with lazy linking change
On 11/04/2011 01:46 PM, Matt Pharr wrote:> r143524 changed ModuleLinker such that when linking a Module B into a > Module A, any internal/private/... functions in B are only linked > into A if there is a use of them in A. This is problematic for my > usage of module linking, where I'm basically linking a few modules > together (some of which include internal functions) and then > generating additional code that may use some of the internal > functions--so I'd like everything to be linked in at the start, so > that all of those functions are available. I don't want to not mark > those functions as internal, since if they're not used, I'd like them > to be eliminated when I'm done with compilation. With this change, > those functions are now no longer available. >This is an interesting regression that also affects LTO. I have reported llvm.org/pr11367 for tracking it. Thanks for finding it!> Thanks, -mattCheers, Rafael