llvm dev - Sep 2013 - [LLVMdev] [lld] Modeling ELF FileNodes/ControlNodes (Group's) in lld

Hi Nick,

On 9/4/2013 4:04 PM, Nick Kledzik wrote:
> I do think we have too many classes.
Agree.
>   I thought InputGraph was going to replace InputFiles.
Interesting idea.
>   It seems link LinkerInput could be merged into FileNode.
Agree.
>
> Originally InputFiles was the abstract interface that he Resolver used to
see all the inputs.  If InputGraph supported the methods  forEachInitalAtom()
and searchLibraries() then we could get rid of InputFiles and have the Resolver
uses InputGraph directly.
Yes, this would be nice. I will try to write a proposal in the next few 
days.
> What should the interface be that the Resolver uses for handling groups?
bool resolveUndefines(std::vector<File> &files) ?

This has to iterate over the files until it reaches a stable point (that 
no more resolution is possible).

Thanks

Shankar Easwaran
>
> -Nick
>
> On Sep 4, 2013, at 1:42 PM, Shankar Easwaran <shankare at
codeaurora.org> wrote:
>
>> Hi,
>>
>> With the inputGraph now, lld models command line options, input files
as nodes in the InputGraph called InputElements.
>>
>> In the current approach, each InputElement is converted to a
LinkerInput, which works if all lld deals with individual files.
>>
>> Dealing with ControlNodes (Groups), have a problem with it, on how to
model that as a LinkerInput.
>>
>> Joerg/Me were chatting on the IRC about this and we came up with the
following approach.
>>
>> - LinkerInput will contain a single file(lld::File), if the node that
its pointing to appears to be a FileNode
>> - LinkerInput will contain a vector(lld::Group) of files(lld::Files) ,
if the node that its pointing appears to be a Group
>>
>> The resolver would need to be modified to consider lld::Groups in
addition to lld::File.
>>
>> Does this sound like the approach we want to take ?
>>
>> Thanks
>>
>> Shankar Easwaran
>>
>> -- 
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation
>
>

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the
Linux Foundation

On Sep 4, 2013, at 2:32 PM, Shankar Easwaran <shankare at codeaurora.org>
wrote:
> On 9/4/2013 4:04 PM, Nick Kledzik wrote:
>> I do think we have too many classes.
> Agree.
>>  I thought InputGraph was going to replace InputFiles.
> Interesting idea.
>>  It seems link LinkerInput could be merged into FileNode.
> Agree.
>> 
>> Originally InputFiles was the abstract interface that he Resolver used
to see all the inputs.  If InputGraph supported the methods  forEachInitalAtom()
and searchLibraries() then we could get rid of InputFiles and have the Resolver
uses InputGraph directly.
> Yes, this would be nice. I will try to write a proposal in the next few
days.
>> What should the interface be that the Resolver uses for handling
groups?
> bool resolveUndefines(std::vector<File> &files) ?
> 
> This has to iterate over the files until it reaches a stable point (that no
more resolution is possible).
It is a little more complicated.  After initial .o files are processed, there
are a bunch of undefines left.  The resolver needs to start loading archive
members that fulfill those undefines, but loading a member can introduce even
more undefines, so it has to iterate.

I think the existing searchLibraries(StringRef name, bool options..) interface
almost works for InputGraph.  To support groups we just need to keep track of
where in the InputGraph we currently are, so that searchLibraries() can spin in
a group until no more undefs are fulfilled, then move on in the graph.

-Nick

>> On Sep 4, 2013, at 1:42 PM, Shankar Easwaran <shankare at
codeaurora.org> wrote:
>> 
>>> Hi,
>>> 
>>> With the inputGraph now, lld models command line options, input
files as nodes in the InputGraph called InputElements.
>>> 
>>> In the current approach, each InputElement is converted to a
LinkerInput, which works if all lld deals with individual files.
>>> 
>>> Dealing with ControlNodes (Groups), have a problem with it, on how
to model that as a LinkerInput.
>>> 
>>> Joerg/Me were chatting on the IRC about this and we came up with
the following approach.
>>> 
>>> - LinkerInput will contain a single file(lld::File), if the node
that its pointing to appears to be a FileNode
>>> - LinkerInput will contain a vector(lld::Group) of
files(lld::Files) , if the node that its pointing appears to be a Group
>>> 
>>> The resolver would need to be modified to consider lld::Groups in
addition to lld::File.
>>> 
>>> Does this sound like the approach we want to take ?
>>> 
>>> Thanks
>>> 
>>> Shankar Easwaran
>>> 
>>> -- 
>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation
>> 
>> 
> 
> 
> -- 
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted
by the Linux Foundation

On Wed, Sep 4, 2013 at 3:50 PM, Nick Kledzik <kledzik at apple.com> wrote:
> On Sep 4, 2013, at 2:32 PM, Shankar Easwaran <shankare at
codeaurora.org>
> wrote:
> > On 9/4/2013 4:04 PM, Nick Kledzik wrote:
> >> I do think we have too many classes.
> > Agree.
> >>  I thought InputGraph was going to replace InputFiles.
> > Interesting idea.
> >>  It seems link LinkerInput could be merged into FileNode.
> > Agree.
> >>
> >> Originally InputFiles was the abstract interface that he Resolver
used
> to see all the inputs.  If InputGraph supported the methods
>  forEachInitalAtom() and searchLibraries() then we could get rid of
> InputFiles and have the Resolver uses InputGraph directly.
> > Yes, this would be nice. I will try to write a proposal in the next
few
> days.
> >> What should the interface be that the Resolver uses for handling
groups?
> > bool resolveUndefines(std::vector<File> &files) ?
> >
> > This has to iterate over the files until it reaches a stable point
(that
> no more resolution is possible).
> It is a little more complicated.  After initial .o files are processed,
> there are a bunch of undefines left.  The resolver needs to start loading
> archive members that fulfill those undefines, but loading a member can
> introduce even more undefines, so it has to iterate.
>
> I think the existing searchLibraries(StringRef name, bool options..)
> interface almost works for InputGraph.  To support groups we just need to
> keep track of where in the InputGraph we currently are, so that
> searchLibraries() can spin in a group until no more undefs are fulfilled,
> then move on in the graph.
>
> -Nick
>
The easiest way to handle this is to have the resolver do it directly.
Darwin ld's behavior of looping over everything can be implemented by
putting everything in an implicit group. We could make it more extensible
by factoring out the searching, but I don't really see a need for that
complexity.

- Michael Spencer

>
>
> >> On Sep 4, 2013, at 1:42 PM, Shankar Easwaran <shankare at
codeaurora.org>
> wrote:
> >>
> >>> Hi,
> >>>
> >>> With the inputGraph now, lld models command line options,
input files
> as nodes in the InputGraph called InputElements.
> >>>
> >>> In the current approach, each InputElement is converted to a
> LinkerInput, which works if all lld deals with individual files.
> >>>
> >>> Dealing with ControlNodes (Groups), have a problem with it, on
how to
> model that as a LinkerInput.
> >>>
> >>> Joerg/Me were chatting on the IRC about this and we came up
with the
> following approach.
> >>>
> >>> - LinkerInput will contain a single file(lld::File), if the
node that
> its pointing to appears to be a FileNode
> >>> - LinkerInput will contain a vector(lld::Group) of
files(lld::Files) ,
> if the node that its pointing appears to be a Group
> >>>
> >>> The resolver would need to be modified to consider lld::Groups
in
> addition to lld::File.
> >>>
> >>> Does this sound like the approach we want to take ?
> >>>
> >>> Thanks
> >>>
> >>> Shankar Easwaran
> >>>
> >>> --
> >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora
Forum,
> hosted by the Linux Foundation
> >>
> >>
> >
> >
> > --
> > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> hosted by the Linux Foundation
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130904/322424fb/attachment.html>

Hi Nick,

These are the below modifications needed in lld to start processing 
groups :-

1) LinkerInput would be moved to FileNode that contains the following 
functions
     -  getBuffer
      - takeBuffer
      - getPath

2) The driver will process the vector of InputElements and call 
/*process */on each of them.
      process() would create a lld::File object within the InputElement 
if its a FileNode
      process() would create a vector of lld::File objects within the 
InputElement if its a ControlNode

3) The resolver will not process each file but it would process 
InputElements and would do the following :-
     3.1 ) if the InputElement corresponds to a fileNode, the resolver 
will call in
             inputElement->processAtoms(*this) and marks the 
InputElement as already processed.
*This would essentially call resolver.processFile(_currentFile)*

      3.2) If the inputElement corresponds to a GroupNode, the resolver 
will call inputElement->processAtom(*this) on
*This* *would essentiall call  
resolver.processGroup(std::vector<Files>), the vector of files is what 
makes it in the current group*

4) InputFiles would be removed
5) LinkerInput would be removed
6) Functions that are unused forEachInitialAtom etc

_*Issues*_*

*This will essentially break the Darwin model thats in lld, as the 
current functionalities processes all the object files and only then 
processes
archive files.

Can we control by using a resolution policy ? which is a set of boolean 
flags to control the resolution behavior ? Do you have any other approach ?

_*Questions*_

1) There might be other types of control nodes, what would we need to 
do, can the resolver be controlled by each flavor ?

Thanks

Shankar Easwaran


On 9/4/2013 5:50 PM, Nick Kledzik wrote:
> On Sep 4, 2013, at 2:32 PM, Shankar Easwaran <shankare at
codeaurora.org> wrote:
>> On 9/4/2013 4:04 PM, Nick Kledzik wrote:
>>> I do think we have too many classes.
>> Agree.
>>>   I thought InputGraph was going to replace InputFiles.
>> Interesting idea.
>>>   It seems link LinkerInput could be merged into FileNode.
>> Agree.
>>> Originally InputFiles was the abstract interface that he Resolver
used to see all the inputs.  If InputGraph supported the methods 
forEachInitalAtom() and searchLibraries() then we could get rid of InputFiles
and have the Resolver uses InputGraph directly.
>> Yes, this would be nice. I will try to write a proposal in the next few
days.
>>> What should the interface be that the Resolver uses for handling
groups?
>> bool resolveUndefines(std::vector<File> &files) ?
>>
>> This has to iterate over the files until it reaches a stable point
(that no more resolution is possible).
> It is a little more complicated.  After initial .o files are processed,
there are a bunch of undefines left.  The resolver needs to start loading
archive members that fulfill those undefines, but loading a member can introduce
even more undefines, so it has to iterate.
>
> I think the existing searchLibraries(StringRef name, bool options..)
interface almost works for InputGraph.  To support groups we just need to keep
track of where in the InputGraph we currently are, so that searchLibraries() can
spin in a group until no more undefs are fulfilled, then move on in the graph.
>
> -Nick
>
>
>>> On Sep 4, 2013, at 1:42 PM, Shankar Easwaran <shankare at
codeaurora.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> With the inputGraph now, lld models command line options, input
files as nodes in the InputGraph called InputElements.
>>>>
>>>> In the current approach, each InputElement is converted to a
LinkerInput, which works if all lld deals with individual files.
>>>>
>>>> Dealing with ControlNodes (Groups), have a problem with it, on
how to model that as a LinkerInput.
>>>>
>>>> Joerg/Me were chatting on the IRC about this and we came up
with the following approach.
>>>>
>>>> - LinkerInput will contain a single file(lld::File), if the
node that its pointing to appears to be a FileNode
>>>> - LinkerInput will contain a vector(lld::Group) of
files(lld::Files) , if the node that its pointing appears to be a Group
>>>>
>>>> The resolver would need to be modified to consider lld::Groups
in addition to lld::File.
>>>>
>>>> Does this sound like the approach we want to take ?
>>>>
>>>> Thanks
>>>>
>>>> Shankar Easwaran
>>>>
>>>> -- 
>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora
Forum, hosted by the Linux Foundation
>>>
>>
>> -- 
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation
>
>

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the
Linux Foundation

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130904/6469ffc9/attachment.html>