llvm dev - Jul 2015 - [LLVMdev] [LLD] support for dlltool generated libs in COFF/PECOFF

>
> For example the gnu linker provides the section with the stub code and the
> offset for injection.
Meant to say dlltool here not gnu linker :)

On Sat, Jul 25, 2015 at 11:17 AM, Martell Malone <martellmalone at
gmail.com>
wrote:
> Hey guys,
>
> So I was able to modify dlltool to produce the exact same layout as
> lib.exe with the same section numbers etc.
> I've first managed to first create the correct section so that lld
gives
> me link errors and then resolve those errors to create an exe.
>
> This is one thing that is still missing that sticks out like a sore thumb
> In the actual imports of the functions the objects are very different
>
> As you can see lib.exe generates a very simple function import object.
>
>   IMPORT_OBJECT[1-N]
>     -> contains an IMPORT_OBJECT_HEADER and DATA
>     data example -> _MessageBoxA.USER32.dll
>
> dlltool however doesn't create an import object but infact has a
complete
> IMAGE_FILE
>
>   IMPORT_OBJECT[1-N]
>     -> contains an IMAGE_FILE_HEADER and is a full object with SECTIONS
>     the .text section has the stub code.
>     the .idata$6 section has the name of the function
>     it starts with 0x01 0x4D then MessageBoxA without an underscore
>     The dll is not referenced and is assumed to be the one listed in the
> first object in .idata$6
>
>
> This is where the actual problem lies because the dll functions never get
> resolved
> Have you guys got any thoughts on weither we should support this or not
> within lld and if so how ?
>
> There are some interesting things here
> For example the gnu linker provides the section with the stub code and the
> offset for injection.
> Should we be using the stub code provided in the text section or should we
> just ignore it and use the one in lld
> They are the same anyway but its just interesting how the dlltool version
> gives us that option.
>
> Kind Regards
> Martell
>
>
> On Fri, Jul 24, 2015 at 10:52 AM, Martell Malone <martellmalone at
gmail.com>
> wrote:
>
>> After some more digging and creating a few testcases in lld I have
>> narrowed it down to
>>
>> The fact that dlltool generates
>>
>> Contents of section .idata$7:
>>  0000 55534552 33322e64 6c6c0000           USER32.dll..
>>
>> Where as lld expects
>>
>> Contents of section .idata$6:
>>  0000 55534552 33322e64 6c6c0000           USER32.dll..
>>
>> I recreated the hello64.test using dlltool for the lib and here is the
asm dump of the final exe
>>
>> hello64gnu.exe:      file format COFF-x86-64
>>
>> Disassembly of section .text:
>> .text:
>>     3000:       ff 25 26 f0 ff ff       jmpq    *-4058(%rip)
>>     3006:       90      nop
>>     3007:       90      nop
>>     3008:       ff 25 26 f0 ff ff       jmpq    *-4058(%rip)
>>     300e:       90      nop
>>     300f:       90      nop
>>     3010:       48 83 ec 28     subq    $40, %rsp
>>     3014:       48 c7 c1 00 00 00 00    movq    $0, %rcx
>>     301b:       48 8d 15 e4 df ff ff    leaq    -8220(%rip), %rdx
>>     3022:       4c 8d 05 d7 df ff ff    leaq    -8233(%rip), %r8
>>     3029:       41 b9 00 00 00 00       movl    $0, %r9d
>>     302f:       e8 d4 ff ff ff  callq   -44
>>     3034:       b9 00 00 00 00  movl    $0, %ecx
>>     3039:       e8 c2 ff ff ff  callq   -62
>>
>> So I am fairly certain this is the cause.
>>
>>
>>
>> Many Thanks
>>
>> Martell
>>
>>
>> On Thu, Jul 23, 2015 at 4:22 PM, Martell Malone <martellmalone at
gmail.com>
>> wrote:
>>
>>> I forgot to attach the notes.txt with the objdump.
>>>
>>>
>>> On Thu, Jul 23, 2015 at 3:55 PM, Martell Malone <martellmalone
at gmail.com
>>> > wrote:
>>>
>>>> Hi again rui, :)
>>>>
>>>> I've got all the patches into llvm and clang for supporting
mingw-w64
>>>> via compiler-rt and now we are able to build a full mingw-w64
toolchain
>>>> without gcc :)
>>>> With great help from yaron and rnk.
>>>>
>>>> I've CC'd them as they might have interest in seeing
this target
>>>> through with me to the end :)
>>>>
>>>> So I have again turned my attention to LLD so that we can also
remove
>>>> ld as a depend for this target
>>>>
>>>> Previously I raised the issue that while lld would infact link
exe's
>>>> for me the issue was the sections were malformed and thus
crashed on launch.
>>>>
>>>> I've done some debugging reguarding this and discovered it
is due to
>>>> differences between what MSVC's lib.exe and binutils
dlltool produces
>>>>
>>>> I have attached a very simple .def file and generated libs via
dlltool
>>>> along with an objdump -s log of the lib.exe version
>>>>
>>>> I don't know too much about the internals of lld to be able
to tell the
>>>> exact cause of the problem but there are 3 specific differences
I noted.
>>>>
>>>> 1. With dlltool each object has the name of a compiled .o file
this is
>>>> the one it generates when building the code section
>>>> With lib each object has the dll name
>>>>
>>>> 2. With dlltool section 7 is used for the dll name and the
function
>>>> names with lib section 6 is used
>>>>
>>>> 3. Other differences are that there is no debug section with
dlltool
>>>> and the text section comes first rather than last with lib
(that probably
>>>> doesn't matter much though)
>>>>
>>>> Please see attached notes.txt with objdump's of both libs
>>>>
>>>> Can could you give me your thoughts on this and how best we
support
>>>> both layouts in COFF/PECOFF ?
>>>>
>>>> What source files should I be looking at to patch this or is
this
>>>> something you can support with very little effort ?
>>>>
>>>> I don't know if you would be opposed to supporting this or
not being
>>>> honest dlltool should generate libs using the same section
number as
>>>> lib.exe but we can't exactly change that now because
existing toolchains
>>>> and libs have already been built with this and changing that
would break
>>>> them.
>>>>
>>>> If you are opposed I would like you to still tell me what I
have to
>>>> change to get it working after I do that I can narrow the scope
for getting
>>>> a patch up streamed into binutils to fix this :)
>>>>
>>>> Kind Regards
>>>> Martell
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150725/7627bd8d/attachment.html>

Hi Martell.

I too was trying to make the new COFF linker recognize the libraries
generated by binutils, but it wasn't resolving the DLLs, just like you
described. Apparently it is not identifying the libraries as import
libraries but archives.

For testing purposes, i've managed to make the old COFF linker work with a
simple test case. Had to make two little changes, and link the exe
manually, because the linker doesn't understand the GNU flags, but it
works. By the way, even the MS linker works with the binutils generated
import libraries.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150725/d12e9aa5/attachment.html>

Hi Ivan, :)

For testing purposes, I've managed to make the old COFF linker work with
a
> simple test case
>
Have you got patches for this that I can look at ?

Had to make two little changes, and link the exe manually, because
the
> linker doesn't understand the GNU flags
>
I've solved the issue of having a GNU driver for the linker so that's
not
an issue anymore
I sent in patches to the mailing list on this previously :)
I keep them updated over in the msys2 project btw

By the way, even the MS linker works with the binutils generated
import
> libraries.
>
Yes i assume binutils was based from a very old version of a MS linker
which did this layout at the time.
lld is so much newer and the MS linker has been updated to a more modern
layout which is why we have this issue

Kind Regards
Martell


On Sat, Jul 25, 2015 at 12:43 PM, Ivan Garramona <heavenandhell171 at
gmail.com
> wrote:
> Hi Martell.
>
> I too was trying to make the new COFF linker recognize the libraries
> generated by binutils, but it wasn't resolving the DLLs, just like you
> described. Apparently it is not identifying the libraries as import
> libraries but archives.
>
> For testing purposes, i've managed to make the old COFF linker work
with a
> simple test case. Had to make two little changes, and link the exe
> manually, because the linker doesn't understand the GNU flags, but it
> works. By the way, even the MS linker works with the binutils generated
> import libraries.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150725/cc6541e4/attachment.html>