llvm dev - Jan 2015 - [LLVMdev] Can we establish layering for the LLD libraries? Current state is a bit of a mess...

On Tue, Jan 20, 2015 at 6:06 PM, Rui Ueyama <ruiu at google.com> wrote:
> On Tue, Jan 20, 2015 at 5:42 PM, Chandler Carruth <chandlerc at
google.com>
> wrote:
>
>>
>> On Tue, Jan 20, 2015 at 5:35 PM, Nick Kledzik <kledzik at
apple.com> wrote:
>>
>>> On Jan 19, 2015, at 6:33 PM, Chandler Carruth <chandlerc at
google.com>
>>> wrote:
>>>
>>> > I wanted to go through and map out the layering of LLD's
libraries
>>> today and found that it's essentially impossible. I think some
serious
>>> cleanup is needed here.
>>> >
>>> > Let's start with the purely link-level dependencies
encoded in the
>>> CMake build:
>>> >
>>> > Curently the Core library depends on the ReaderWriter/Native
library,
>>> which links against the ReaderWriter library, which links against
the Core
>>> library. This clearly cannot work. The same cycle exists with Core
-> YAML
>>> -> ReaderWrite -> Core.
>>> How are you determining these cycles?   How is Core dependent on
YAML?
>>> (cause that seems wrong).
>>>
>>
>> This is in the CMake build system today. You can see it with
>> CMakeLists.txt. Also you can look at Greg's patch for some of the
issues
>> here.
>>
>
> It seems the dependency to lldYAML can be removed from CMakeList.txt now.
> Core doesn't actually depends on YAML.
>
> lldPasses depends on libNative, and that dependency is currently resolved
> through lldCore transitively. We should be able to remove the dependency to
> libNative from Core by fixing Pass's CMake file.
>
Cool, this makes lots of sense to me as well. We can probably fix these
minor issues quickly.

Still, it would be good to get a firm big picture that everyone agrees to
in place.

>
> If my understanding is correct, the dependencies dependencies between
> components in LLD should be like this:
>
> Config: (nothing)
> Core: Config
> Driver: Core Passes ReaderWriter
> Passes: Core ReaderWriter
> ReaderWriter: Core (and Driver?)
>
> I don't want ReaderWriter to depend on Driver, but it may be
unavoidable
> because of .drectve section in PE/COFF which contains command line options.
>
Well, it *can't* depend on Driver if Driver depends on ReaderWriter.

>
> ReaderWriter should be able to be split into smaller libraries because
> there's no cross-arch dependency between sub-directories in
ReaderWriter.
>
That would be nice. Can you sketch out what this would look like?


The other issue is that while the above makes sense to me, if you look at
the #include lines in these libraries, it is *very* far from the truth. How
do we fix this? In particular, the LinkingContext seems to establish a lot
of circular dependencies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150120/a18dc6b7/attachment.html>

On Tue, Jan 20, 2015 at 6:10 PM, Chandler Carruth <chandlerc at
google.com>
wrote:
>
> On Tue, Jan 20, 2015 at 6:06 PM, Rui Ueyama <ruiu at google.com>
wrote:
>
>> On Tue, Jan 20, 2015 at 5:42 PM, Chandler Carruth <chandlerc at
google.com>
>> wrote:
>>
>>>
>>> On Tue, Jan 20, 2015 at 5:35 PM, Nick Kledzik <kledzik at
apple.com> wrote:
>>>
>>>> On Jan 19, 2015, at 6:33 PM, Chandler Carruth <chandlerc at
google.com>
>>>> wrote:
>>>>
>>>> > I wanted to go through and map out the layering of
LLD's libraries
>>>> today and found that it's essentially impossible. I think
some serious
>>>> cleanup is needed here.
>>>> >
>>>> > Let's start with the purely link-level dependencies
encoded in the
>>>> CMake build:
>>>> >
>>>> > Curently the Core library depends on the
ReaderWriter/Native library,
>>>> which links against the ReaderWriter library, which links
against the Core
>>>> library. This clearly cannot work. The same cycle exists with
Core -> YAML
>>>> -> ReaderWrite -> Core.
>>>> How are you determining these cycles?   How is Core dependent
on YAML?
>>>> (cause that seems wrong).
>>>>
>>>
>>> This is in the CMake build system today. You can see it with
>>> CMakeLists.txt. Also you can look at Greg's patch for some of
the issues
>>> here.
>>>
>>
>> It seems the dependency to lldYAML can be removed from CMakeList.txt
now.
>> Core doesn't actually depends on YAML.
>>
>> lldPasses depends on libNative, and that dependency is currently
resolved
>> through lldCore transitively. We should be able to remove the
dependency to
>> libNative from Core by fixing Pass's CMake file.
>>
>
> Cool, this makes lots of sense to me as well. We can probably fix these
> minor issues quickly.
>
> Still, it would be good to get a firm big picture that everyone agrees to
> in place.
>
>
>>
>> If my understanding is correct, the dependencies dependencies between
>> components in LLD should be like this:
>>
>> Config: (nothing)
>> Core: Config
>> Driver: Core Passes ReaderWriter
>> Passes: Core ReaderWriter
>> ReaderWriter: Core (and Driver?)
>>
>> I don't want ReaderWriter to depend on Driver, but it may be
unavoidable
>> because of .drectve section in PE/COFF which contains command line
options.
>>
>
> Well, it *can't* depend on Driver if Driver depends on ReaderWriter.
>
Oh that's true. Then the only thing we can do is to combine Driver and
ReaderWriter to make it one library? It's far from ideal and feels wrong
though.

>> ReaderWriter should be able to be split into smaller libraries because
>> there's no cross-arch dependency between sub-directories in
ReaderWriter.
>>
>
> That would be nice. Can you sketch out what this would look like?
>
Basically each sub-directory in lib/ReaderWriter should be able to become a
separate library.

>
>
> The other issue is that while the above makes sense to me, if you look at
> the #include lines in these libraries, it is *very* far from the truth. How
> do we fix this? In particular, the LinkingContext seems to establish a lot
> of circular dependencies.
>
Well that's not easy to answer without taking a closer look at the code,
but I agree that the currently #include dependencies are nasty.

One idea would be to move LinkingContext.h to Core while keeping all
arch-specific LinkingContexts in ReaderWriter. I believe Core and Passes
depends only on (generic) LinkingContext so we can remove dependency from
Core/Passes to ReaderWriter by doing that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150120/433b1054/attachment.html>

On Jan 20, 2015, at 6:37 PM, Rui Ueyama <ruiu at google.com> wrote:
> If my understanding is correct, the dependencies dependencies between
components in LLD should be like this:
> 
> Config: (nothing)
> Core: Config
> Driver: Core Passes ReaderWriter
> Passes: Core ReaderWriter
> ReaderWriter: Core (and Driver?)
> 
> I don't want ReaderWriter to depend on Driver, but it may be
unavoidable because of .drectve section in PE/COFF which contains command line
options.
> 
> Well, it *can't* depend on Driver if Driver depends on ReaderWriter.
> 
> Oh that's true. Then the only thing we can do is to combine Driver and
ReaderWriter to make it one library? It's far from ideal and feels wrong
though.
Can we finesse this by having the WinLinkDriver register a “parse” function
pointer with PECOFFLinkingContext.  Then have FileCOFF just call the registered
parse function in the PECOFFLinkingContext?

-Nick


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150120/20b8eabf/attachment.html>