Okay, I understood that you're proposing to add all undefined symbols during the resolution step, and not try to collect extra symbols during execution and then check if some undefines left (as I originally planned). This sounds reasonable as in any case we must have all undefines resolved in order to continue the linking process. Concerning the implementation, why not to add this virtual archive file to the OutputELFWriter (or even to ExecutableWriter) since we already have a method to add specific files to the linking process? We may then expose a simple interface to the descendants of the writers to give chance to handle undefines. Also, do we need this special symbol handling for any cases other than static linking of the executable? - Denis. On 02/24/2015 06:44 AM, Nick Kledzik wrote: On Feb 23, 2015, at 2:52 PM, Michael Spencer <bigcheesegs at gmail.com><mailto:bigcheesegs at gmail.com> wrote:> On Thu, Feb 19, 2015 at 10:40 PM, Denis Protivensky > <dprotivensky at accesssoftek.com><mailto:dprotivensky at accesssoftek.com> wrote: >> Shankar, >> >> Okay, I guessed the correct interface. >> But what about the moment at which the function is called? >> If it's called from Resolver::resolve(), it doesn't make any difference to >> me as I cannot determine the need of specific symbols at that time. >> >> - Denis. > > None of the symbols we are looking up require the full resolver, and > they are all special linker symbols. I propose two things. > > 1. Provide a hook as per what Shankar suggested for the resolver. User > references to linker defined symbols such as _GLOBAL_OFFSET_TABLE_ get > created and possibly deadstripped here. The linking context owns the > atom. > 2. The ELFLinkingContext gains <Atom > *getOrCreateLinkerDefinedAtom(StringRef);>. This can be used in passes > to get the symbols. The hook in (1) would call this to create the > atoms. > > This gives a single place where linker defined atoms are actually > created, and allows correct deadstripping and object file references > without doing multiple resolver passes.As Rui showed, we already have this abstraction. The linking context just adds a magic ArchiveFile. When queried for any “linker defined symbol”, the magic ArchiveFile instantiates the atoms needed. This is how mach-o handles linker defined symbols like __dso_handle. -Nick> >> >> >> On 02/19/2015 08:15 PM, Shankar Easwaran wrote: >> >> + Nick >> >> On 2/19/2015 9:00 AM, Shankar Easwaran wrote: >>> On 2/19/2015 3:58 AM, Denis Protivensky wrote: >>>> Joerg: >>>>> I propose to add the ability to ignore undefined symbols during initial >>>>> resolution, and then postprocess only those undefines for the second >>>>> time >>>>> after the pass manager execution. >>>> Do you want to do that before or after dead code elimination? >>>> I think dead code elimination should be performed after all possible >>>> object code modifications done by lld. Therefore, it should be done >>>> after undefines' postprocessing as well. >>> Gnu does dead code elimination before undefines are reported. So if a >>> function is not called and it has a undefined reference its would not >>> be an undef. >>>> >>>> Shankar: >>>>> I propose to add the ability to ignore undefined symbols during initial >>>>> resolution, and then postprocess only those undefines for the second >>>>> time >>>>> after the pass manager execution. >>>> I came across this same problem, and was planning on adding a >>>> notifyUndefinedSymbol to the LinkingContext, if the linker wants to add >>>> a defined symbol and coalesce it, it would be possible. >>>> >>>> Do you think this will work for your case too ? >>>> With this option, I don't see: >>>> - how to postpone processing and reaction on undefines. If the >>>> callback is called from within Resolver::resolve(), you should react >>>> on it immediately, because otherwise the code will still fail in >>>> Resolver::resolve(). >>>> - how to know if a symbol is needed within the callback body. The >>>> need of any symbol is determined in some other place. So I need to >>>> keep a sort of indication (boolean flags, whatever) to know which >>>> symbols are really needed. >>>> - the exact interface of notifyUndefinedSymbol callback. If it >>>> receives `StringRef` name of the undefined symbol, what reaction >>>> should be? Should it return new symbols to add back to the caller as >>>> `const Atom*`? >>> notifyUndefinedSymbol will allow the context to coalesce the undefined >>> atom with a defined atom. >>> >>> Atom *notifyUndefinedSymbol(StringRef name) could be the interface. >>> >>>> Thanks, >>>> Denis. >>>> >>> >>> >> >> >> -- >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by >> the Linux Foundation >> >> >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150225/a5f47137/attachment.html>
Adding it to the OutputELFWriter sounds good. On 2/25/2015 6:40 AM, Denis Protivensky wrote:> Okay, I understood that you're proposing to add all undefined symbols during the resolution step, and not try to collect extra symbols during execution and then check if some undefines left (as I originally planned). > This sounds reasonable as in any case we must have all undefines resolved in order to continue the linking process. > > Concerning the implementation, why not to add this virtual archive file to the OutputELFWriter (or even to ExecutableWriter) since we already have a method to add specific files to the linking process? > We may then expose a simple interface to the descendants of the writers to give chance to handle undefines. > > Also, do we need this special symbol handling for any cases other than static linking of the executable? > > - Denis. > > On 02/24/2015 06:44 AM, Nick Kledzik wrote: > > > On Feb 23, 2015, at 2:52 PM, Michael Spencer <bigcheesegs at gmail.com><mailto:bigcheesegs at gmail.com> wrote: > >> On Thu, Feb 19, 2015 at 10:40 PM, Denis Protivensky >> <dprotivensky at accesssoftek.com><mailto:dprotivensky at accesssoftek.com> wrote: >>> Shankar, >>> >>> Okay, I guessed the correct interface. >>> But what about the moment at which the function is called? >>> If it's called from Resolver::resolve(), it doesn't make any difference to >>> me as I cannot determine the need of specific symbols at that time. >>> >>> - Denis. >> None of the symbols we are looking up require the full resolver, and >> they are all special linker symbols. I propose two things. >> >> 1. Provide a hook as per what Shankar suggested for the resolver. User >> references to linker defined symbols such as _GLOBAL_OFFSET_TABLE_ get >> created and possibly deadstripped here. The linking context owns the >> atom. >> 2. The ELFLinkingContext gains <Atom >> *getOrCreateLinkerDefinedAtom(StringRef);>. This can be used in passes >> to get the symbols. The hook in (1) would call this to create the >> atoms. >> >> This gives a single place where linker defined atoms are actually >> created, and allows correct deadstripping and object file references >> without doing multiple resolver passes. > As Rui showed, we already have this abstraction. The linking context just adds a magic ArchiveFile. When queried for any “linker defined symbol”, the magic ArchiveFile instantiates the atoms needed. > > This is how mach-o handles linker defined symbols like __dso_handle. > > -Nick > > >>> >>> On 02/19/2015 08:15 PM, Shankar Easwaran wrote: >>> >>> + Nick >>> >>> On 2/19/2015 9:00 AM, Shankar Easwaran wrote: >>>> On 2/19/2015 3:58 AM, Denis Protivensky wrote: >>>>> Joerg: >>>>>> I propose to add the ability to ignore undefined symbols during initial >>>>>> resolution, and then postprocess only those undefines for the second >>>>>> time >>>>>> after the pass manager execution. >>>>> Do you want to do that before or after dead code elimination? >>>>> I think dead code elimination should be performed after all possible >>>>> object code modifications done by lld. Therefore, it should be done >>>>> after undefines' postprocessing as well. >>>> Gnu does dead code elimination before undefines are reported. So if a >>>> function is not called and it has a undefined reference its would not >>>> be an undef. >>>>> Shankar: >>>>>> I propose to add the ability to ignore undefined symbols during initial >>>>>> resolution, and then postprocess only those undefines for the second >>>>>> time >>>>>> after the pass manager execution. >>>>> I came across this same problem, and was planning on adding a >>>>> notifyUndefinedSymbol to the LinkingContext, if the linker wants to add >>>>> a defined symbol and coalesce it, it would be possible. >>>>> >>>>> Do you think this will work for your case too ? >>>>> With this option, I don't see: >>>>> - how to postpone processing and reaction on undefines. If the >>>>> callback is called from within Resolver::resolve(), you should react >>>>> on it immediately, because otherwise the code will still fail in >>>>> Resolver::resolve(). >>>>> - how to know if a symbol is needed within the callback body. The >>>>> need of any symbol is determined in some other place. So I need to >>>>> keep a sort of indication (boolean flags, whatever) to know which >>>>> symbols are really needed. >>>>> - the exact interface of notifyUndefinedSymbol callback. If it >>>>> receives `StringRef` name of the undefined symbol, what reaction >>>>> should be? Should it return new symbols to add back to the caller as >>>>> `const Atom*`? >>>> notifyUndefinedSymbol will allow the context to coalesce the undefined >>>> atom with a defined atom. >>>> >>>> Atom *notifyUndefinedSymbol(StringRef name) could be the interface. >>>> >>>>> Thanks, >>>>> Denis. >>>>> >>>> >>> >>> -- >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by >>> the Linux Foundation >>> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150225/fcefd415/attachment.html>
Shankar, Back when we started the discussion, you mentioned that you also wanted to take care of this case. Will you do that or otherwise I'll work on it? - Denis. On 02/25/2015 05:25 PM, Shankar Easwaran wrote: Adding it to the OutputELFWriter sounds good. On 2/25/2015 6:40 AM, Denis Protivensky wrote: Okay, I understood that you're proposing to add all undefined symbols during the resolution step, and not try to collect extra symbols during execution and then check if some undefines left (as I originally planned). This sounds reasonable as in any case we must have all undefines resolved in order to continue the linking process. Concerning the implementation, why not to add this virtual archive file to the OutputELFWriter (or even to ExecutableWriter) since we already have a method to add specific files to the linking process? We may then expose a simple interface to the descendants of the writers to give chance to handle undefines. Also, do we need this special symbol handling for any cases other than static linking of the executable? - Denis. On 02/24/2015 06:44 AM, Nick Kledzik wrote: On Feb 23, 2015, at 2:52 PM, Michael Spencer <bigcheesegs at gmail.com><mailto:bigcheesegs at gmail.com><mailto:bigcheesegs at gmail.com><mailto:bigcheesegs at gmail.com> wrote: On Thu, Feb 19, 2015 at 10:40 PM, Denis Protivensky <dprotivensky at accesssoftek.com><mailto:dprotivensky at accesssoftek.com><mailto:dprotivensky at accesssoftek.com><mailto:dprotivensky at accesssoftek.com> wrote: Shankar, Okay, I guessed the correct interface. But what about the moment at which the function is called? If it's called from Resolver::resolve(), it doesn't make any difference to me as I cannot determine the need of specific symbols at that time. - Denis. None of the symbols we are looking up require the full resolver, and they are all special linker symbols. I propose two things. 1. Provide a hook as per what Shankar suggested for the resolver. User references to linker defined symbols such as _GLOBAL_OFFSET_TABLE_ get created and possibly deadstripped here. The linking context owns the atom. 2. The ELFLinkingContext gains <Atom *getOrCreateLinkerDefinedAtom(StringRef);>. This can be used in passes to get the symbols. The hook in (1) would call this to create the atoms. This gives a single place where linker defined atoms are actually created, and allows correct deadstripping and object file references without doing multiple resolver passes. As Rui showed, we already have this abstraction. The linking context just adds a magic ArchiveFile. When queried for any “linker defined symbol”, the magic ArchiveFile instantiates the atoms needed. This is how mach-o handles linker defined symbols like __dso_handle. -Nick On 02/19/2015 08:15 PM, Shankar Easwaran wrote: + Nick On 2/19/2015 9:00 AM, Shankar Easwaran wrote: On 2/19/2015 3:58 AM, Denis Protivensky wrote: Joerg: I propose to add the ability to ignore undefined symbols during initial resolution, and then postprocess only those undefines for the second time after the pass manager execution. Do you want to do that before or after dead code elimination? I think dead code elimination should be performed after all possible object code modifications done by lld. Therefore, it should be done after undefines' postprocessing as well. Gnu does dead code elimination before undefines are reported. So if a function is not called and it has a undefined reference its would not be an undef. Shankar: I propose to add the ability to ignore undefined symbols during initial resolution, and then postprocess only those undefines for the second time after the pass manager execution. I came across this same problem, and was planning on adding a notifyUndefinedSymbol to the LinkingContext, if the linker wants to add a defined symbol and coalesce it, it would be possible. Do you think this will work for your case too ? With this option, I don't see: - how to postpone processing and reaction on undefines. If the callback is called from within Resolver::resolve(), you should react on it immediately, because otherwise the code will still fail in Resolver::resolve(). - how to know if a symbol is needed within the callback body. The need of any symbol is determined in some other place. So I need to keep a sort of indication (boolean flags, whatever) to know which symbols are really needed. - the exact interface of notifyUndefinedSymbol callback. If it receives `StringRef` name of the undefined symbol, what reaction should be? Should it return new symbols to add back to the caller as `const Atom*`? notifyUndefinedSymbol will allow the context to coalesce the undefined atom with a defined atom. Atom *notifyUndefinedSymbol(StringRef name) could be the interface. Thanks, Denis. -- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation _______________________________________________ LLVM Developers mailing list LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu><mailto:LLVMdev at cs.uiuc.edu><mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev _______________________________________________ LLVM Developers mailing list LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu><mailto:LLVMdev at cs.uiuc.edu><mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev _______________________________________________ LLVM Developers mailing list LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev -- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150302/31e09ad3/attachment.html>