Martell Malone via llvm-dev
2017-Feb-14 00:46 UTC
[llvm-dev] RFC: A new llvm-dlltool driver and llvm-lib driver improvements
> > No, I meant an even thinner wrapper which textually translates arguments. > For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o > foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything > with Config object nor LinkerDriver::run and have absolutely zero knowledge > on the internals of LLD.Ohh okay I misunderstood. (Why I'm asking this is because I want to sure that there's no difference> between the regular Windows linker and mingw. If all differences are > superficial, the command line translator should just work. If not, there's > some semantic difference.)The libraries can be used with MSVC link.exe directly assuming the correct arguments are passed to the linker. I tested this after creating a working llvm-dlltool. The only change I had to make to support this was alter ming-w64 crt to change all references from __image_base__ to _ImageBase to support that That should be fine. Great On Mon, Feb 13, 2017 at 9:34 PM, Rui Ueyama <ruiu at google.com> wrote:> On Mon, Feb 13, 2017 at 1:16 PM, Martell Malone <martellmalone at gmail.com> > wrote: > >> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process >>> wrapper, meaning that you could add a main function for mingw which >>> translates all arguments into the MSVC style and then invoke the COFF >>> linker's main function. >> >> That should work, if I am understanding this correctly I can create an >> argument parser (probably partially based on the ELF one) then I can set >> the data in the COFF Config object and call COFF LinkerDriver::run etc >> directly from that? >> > > No, I meant an even thinner wrapper which textually translates arguments. > For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o > foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything > with Config object nor LinkerDriver::run and have absolutely zero knowledge > on the internals of LLD. > > (Why I'm asking this is because I want to sure that there's no difference > between the regular Windows linker and mingw. If all differences are > superficial, the command line translator should just work. If not, there's > some semantic difference.) > > This proposal is for generally moving code from lld into llvm that could >> be part of lib.exe / dlltool, what are your thoughts here? >> > > That should be fine. > > >> On Mon, Feb 13, 2017 at 8:57 PM, Rui Ueyama <ruiu at google.com> wrote: >> >>> On Mon, Feb 13, 2017 at 12:52 PM, Martell Malone < >>> martellmalone at gmail.com> wrote: >>> >>>> Also you need to make a change to LLD/COFF to accept GNU command >>>>> arguments, right? (Looks like you already have that patch locally.) >>>> >>>> Yes >>>> >>> >>> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process >>> wrapper, meaning that you could add a main function for mingw which >>> translates all arguments into the MSVC style and then invoke the COFF >>> linker's main function. >>> >>> >>>> >>>> My patch to hack lld into accepting some very basic gnu front end >>>>> arguments was enough to get all the above working which was enough to >>>>> develop further. >>>> >>>> >>>> >>>> On Mon, Feb 13, 2017 at 8:41 PM, Rui Ueyama <ruiu at google.com> wrote: >>>> >>>>> On Mon, Feb 13, 2017 at 12:33 PM, Martell Malone < >>>>> martellmalone at gmail.com> wrote: >>>>> >>>>>> Hey Rui, >>>>>> >>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw >>>>>>> support. I don't think dlltool is the last missing piece. What do you need >>>>>>> to do other than that to fully support mingw using LLVM toolchain? >>>>>> >>>>>> Other then changing `lib/MC/WinCOFFStreamer.cpp` to not use >>>>>> -aligncomm within the EmitCommonSymbol function and a single patch for >>>>>> mingw-w64 itself to pre-populate it's .ctors and .dtors list, so >>>>>> llvm-dlltool is infact the only missing part really. >>>>>> >>>>> >>>>> Also you need to make a change to LLD/COFF to accept GNU command >>>>> arguments, right? (Looks like you already have that patch locally.) >>>>> >>>>> >>>>>> This gives us a fully working clang based mingw-w64 C compiler. >>>>>> >>>>>> C++ and exception handling is a different story. >>>>>> >>>>>> libc++ is somewhat working with the following test results >>>>>> >>>>>> Expected Passes : 2188 >>>>>> Expected Failures : 44 >>>>>> Unsupported Tests : 588 >>>>>> Unexpected Failures: 2816 >>>>>> >>>>>> I was able to build it with exceptions disabled and I actually >>>>>> managed to bootstrap llvm and clang itself. >>>>>> It didn't run very well though as you can imagine based on the above >>>>>> tests. >>>>>> >>>>>> It's not a performance issue but a code maintenance issue. The >>>>>>> initial patches to support mingw was trying to add a new linker driver and >>>>>>> a different linkin semantics to the COFF linker which seemed too >>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU >>>>>>> command line arguments to MSVC linker arguments. Didn't it work? >>>>>> >>>>>> There were just so many differences between link and ld I never >>>>>> followed down that path. I pushed forward with the short import library >>>>>> support based on your later suggestions. My patch to hack lld into >>>>>> accepting some very basic gnu front end arguments was enough to get all the >>>>>> above working which was enough to develop further. >>>>>> >>>>>> Best, >>>>>> Martell >>>>>> >>>>>> On Mon, Feb 13, 2017 at 8:08 PM, Rui Ueyama <ruiu at google.com> wrote: >>>>>> >>>>>>> Hi Martell, >>>>>>> >>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw >>>>>>> support. I don't think dlltool is the last missing piece. What do you need >>>>>>> to do other than that to fully support mingw using LLVM toolchain? >>>>>>> >>>>>>> On Mon, Feb 13, 2017 at 8:56 AM, Martell Malone via llvm-dev < >>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>> >>>>>>>> Hey llvm'ers, >>>>>>>> >>>>>>>> I have been working on a dlltool replacement for llvm. >>>>>>>> Here is my initial differential https://reviews.llvm.org/D29892 >>>>>>>> It is based on some functionality that already exists in lld. >>>>>>>> I added functionality to support, PE COFF Weak Externals and of >>>>>>>> course a front end to actually use it. >>>>>>>> I believe the work here can also be used for llvm-lib and lessen >>>>>>>> the load on lld. >>>>>>>> I would like some comments about how this could be be structured to >>>>>>>> live in llvm with a shared code base across lib ar and dlltool. >>>>>>>> I also have a section below called "Difference from lib" which is >>>>>>>> somewhat of a rationale for the tool. >>>>>>>> >>>>>>>> Many Thanks, >>>>>>>> Martell >>>>>>>> >>>>>>>> >>>>>>>> Context >>>>>>>> =========>>>>>>>> >>>>>>>> Awhile back I talked to various llvm'ers about getting mingw-w64 >>>>>>>> support for lld. >>>>>>>> There were a few issues raised but the main issue was that >>>>>>>> mingw-w64 should try best to comply with the PECOFF spec, adding support >>>>>>>> for custom sections and various binutils/mingw hacks would impact the >>>>>>>> performance of the COFF linker and in general is not something that lld >>>>>>>> should support. >>>>>>>> >>>>>>> >>>>>>> It's not a performance issue but a code maintenance issue. The >>>>>>> initial patches to support mingw was trying to add a new linker driver and >>>>>>> a different linkin semantics to the COFF linker which seemed too >>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU >>>>>>> command line arguments to MSVC linker arguments. Didn't it work? >>>>>>> >>>>>>> >>>>>>>> Motivation >>>>>>>> =========>>>>>>>> >>>>>>>> The main motivation was because dlltool and ld did not comply with >>>>>>>> PECOFF Spec. >>>>>>>> It has some custom formatting and uses the assembler for some >>>>>>>> reason to generate import libraries, it did not use the short import >>>>>>>> library format at all. >>>>>>>> >>>>>>>> There has been many work arounds for the problem this creates such >>>>>>>> as the creation of the `reimp` tool. Which imports MSVC built libs creates >>>>>>>> a def and uses dlltool so that the binutils linker can use it. >>>>>>>> >>>>>>>> We should just be using the short import format and the linker >>>>>>>> should support that. >>>>>>>> Thus llvm-dlltool was born. >>>>>>>> >>>>>>>> >>>>>>>> Difference from lib >>>>>>>> ==================>>>>>>>> >>>>>>>> Using PE COFF spec (section 8, Import Library Format) should be >>>>>>>> self explanatory. >>>>>>>> lib.exe is able to accept def files and create libraries using this >>>>>>>> format. >>>>>>>> >>>>>>>> example >>>>>>>> >>>>>>>> `LIBRARY "user32.dll" >>>>>>>> EXPORTS >>>>>>>> MessageBoxA` >>>>>>>> >>>>>>>> LIB.exe can create a user32.lib with the function MessageBoxA from >>>>>>>> the above definition. >>>>>>>> >>>>>>>> Mingw-w64 is different MSVC in that we need to compile the runtime. >>>>>>>> MS provide us with their crt prebuilt so lib.exe doesn't have >>>>>>>> support for external function aliasing. >>>>>>>> We often use aliases for posix naming reasons as well as avoid >>>>>>>> using the MS version of a function. >>>>>>>> >>>>>>>> example >>>>>>>> >>>>>>>> `LIBRARY "user32.dll" >>>>>>>> EXPORTS >>>>>>>> MessageBoxA >>>>>>>> MessageBoxW==MessageBoxA` >>>>>>>> >>>>>>>> LIB.exe doesn't not have support for this but dlltool does. >>>>>>>> The best fit for this within the spec is PE COFF spec (Aux Format >>>>>>>> 3: Weak Externals) >>>>>>>> >>>>>>>> Mingw-w64 also uses lib prefix and .a file extensions for the >>>>>>>> library name so the driver should be different. The above example would be >>>>>>>> libuser32.a, for when shared and static versions of the same library exist >>>>>>>> there is the .dll.a variant. >>>>>>>> >>>>>>>> >>>>>>>> Issues >>>>>>>> ======>>>>>>>> >>>>>>>> Like more of the gnu suite dlltool was not designed in one build >>>>>>>> multi target manner. >>>>>>>> We would have to introduce custom arguments to specify the target, >>>>>>>> "i686", "x86_64", "thumb2pe" etc. >>>>>>>> This will most likely break some level of backwards compatibility >>>>>>>> with the binutils version. >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> llvm-dev at lists.llvm.org >>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170214/0aa058ba/attachment.html>
Rui Ueyama via llvm-dev
2017-Feb-14 01:20 UTC
[llvm-dev] RFC: A new llvm-dlltool driver and llvm-lib driver improvements
On Mon, Feb 13, 2017 at 4:46 PM, Martell Malone <martellmalone at gmail.com> wrote:> No, I meant an even thinner wrapper which textually translates arguments. >> For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o >> foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything >> with Config object nor LinkerDriver::run and have absolutely zero knowledge >> on the internals of LLD. > > Ohh okay I misunderstood. > > (Why I'm asking this is because I want to sure that there's no difference >> between the regular Windows linker and mingw. If all differences are >> superficial, the command line translator should just work. If not, there's >> some semantic difference.) > > The libraries can be used with MSVC link.exe directly assuming the correct > arguments are passed to the linker. > I tested this after creating a working llvm-dlltool. > The only change I had to make to support this was alter ming-w64 crt to > change all references from __image_base__ to _ImageBase to support that >You may be able to define __ImageBase as a weak external symbol to __image_base__. Then, if __ImageBase is not defined, all references against __ImageBase will be resolved using __image_base__.> That should be fine. > > Great > > On Mon, Feb 13, 2017 at 9:34 PM, Rui Ueyama <ruiu at google.com> wrote: > >> On Mon, Feb 13, 2017 at 1:16 PM, Martell Malone <martellmalone at gmail.com> >> wrote: >> >>> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process >>>> wrapper, meaning that you could add a main function for mingw which >>>> translates all arguments into the MSVC style and then invoke the COFF >>>> linker's main function. >>> >>> That should work, if I am understanding this correctly I can create an >>> argument parser (probably partially based on the ELF one) then I can set >>> the data in the COFF Config object and call COFF LinkerDriver::run etc >>> directly from that? >>> >> >> No, I meant an even thinner wrapper which textually translates arguments. >> For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o >> foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything >> with Config object nor LinkerDriver::run and have absolutely zero knowledge >> on the internals of LLD. >> >> (Why I'm asking this is because I want to sure that there's no difference >> between the regular Windows linker and mingw. If all differences are >> superficial, the command line translator should just work. If not, there's >> some semantic difference.) >> >> This proposal is for generally moving code from lld into llvm that could >>> be part of lib.exe / dlltool, what are your thoughts here? >>> >> >> That should be fine. >> >> >>> On Mon, Feb 13, 2017 at 8:57 PM, Rui Ueyama <ruiu at google.com> wrote: >>> >>>> On Mon, Feb 13, 2017 at 12:52 PM, Martell Malone < >>>> martellmalone at gmail.com> wrote: >>>> >>>>> Also you need to make a change to LLD/COFF to accept GNU command >>>>>> arguments, right? (Looks like you already have that patch locally.) >>>>> >>>>> Yes >>>>> >>>> >>>> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process >>>> wrapper, meaning that you could add a main function for mingw which >>>> translates all arguments into the MSVC style and then invoke the COFF >>>> linker's main function. >>>> >>>> >>>>> >>>>> My patch to hack lld into accepting some very basic gnu front end >>>>>> arguments was enough to get all the above working which was enough to >>>>>> develop further. >>>>> >>>>> >>>>> >>>>> On Mon, Feb 13, 2017 at 8:41 PM, Rui Ueyama <ruiu at google.com> wrote: >>>>> >>>>>> On Mon, Feb 13, 2017 at 12:33 PM, Martell Malone < >>>>>> martellmalone at gmail.com> wrote: >>>>>> >>>>>>> Hey Rui, >>>>>>> >>>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw >>>>>>>> support. I don't think dlltool is the last missing piece. What do you need >>>>>>>> to do other than that to fully support mingw using LLVM toolchain? >>>>>>> >>>>>>> Other then changing `lib/MC/WinCOFFStreamer.cpp` to not use >>>>>>> -aligncomm within the EmitCommonSymbol function and a single patch for >>>>>>> mingw-w64 itself to pre-populate it's .ctors and .dtors list, so >>>>>>> llvm-dlltool is infact the only missing part really. >>>>>>> >>>>>> >>>>>> Also you need to make a change to LLD/COFF to accept GNU command >>>>>> arguments, right? (Looks like you already have that patch locally.) >>>>>> >>>>>> >>>>>>> This gives us a fully working clang based mingw-w64 C compiler. >>>>>>> >>>>>>> C++ and exception handling is a different story. >>>>>>> >>>>>>> libc++ is somewhat working with the following test results >>>>>>> >>>>>>> Expected Passes : 2188 >>>>>>> Expected Failures : 44 >>>>>>> Unsupported Tests : 588 >>>>>>> Unexpected Failures: 2816 >>>>>>> >>>>>>> I was able to build it with exceptions disabled and I actually >>>>>>> managed to bootstrap llvm and clang itself. >>>>>>> It didn't run very well though as you can imagine based on the above >>>>>>> tests. >>>>>>> >>>>>>> It's not a performance issue but a code maintenance issue. The >>>>>>>> initial patches to support mingw was trying to add a new linker driver and >>>>>>>> a different linkin semantics to the COFF linker which seemed too >>>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU >>>>>>>> command line arguments to MSVC linker arguments. Didn't it work? >>>>>>> >>>>>>> There were just so many differences between link and ld I never >>>>>>> followed down that path. I pushed forward with the short import library >>>>>>> support based on your later suggestions. My patch to hack lld into >>>>>>> accepting some very basic gnu front end arguments was enough to get all the >>>>>>> above working which was enough to develop further. >>>>>>> >>>>>>> Best, >>>>>>> Martell >>>>>>> >>>>>>> On Mon, Feb 13, 2017 at 8:08 PM, Rui Ueyama <ruiu at google.com> wrote: >>>>>>> >>>>>>>> Hi Martell, >>>>>>>> >>>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw >>>>>>>> support. I don't think dlltool is the last missing piece. What do you need >>>>>>>> to do other than that to fully support mingw using LLVM toolchain? >>>>>>>> >>>>>>>> On Mon, Feb 13, 2017 at 8:56 AM, Martell Malone via llvm-dev < >>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>> >>>>>>>>> Hey llvm'ers, >>>>>>>>> >>>>>>>>> I have been working on a dlltool replacement for llvm. >>>>>>>>> Here is my initial differential https://reviews.llvm.org/D29892 >>>>>>>>> It is based on some functionality that already exists in lld. >>>>>>>>> I added functionality to support, PE COFF Weak Externals and of >>>>>>>>> course a front end to actually use it. >>>>>>>>> I believe the work here can also be used for llvm-lib and lessen >>>>>>>>> the load on lld. >>>>>>>>> I would like some comments about how this could be be structured >>>>>>>>> to live in llvm with a shared code base across lib ar and dlltool. >>>>>>>>> I also have a section below called "Difference from lib" which is >>>>>>>>> somewhat of a rationale for the tool. >>>>>>>>> >>>>>>>>> Many Thanks, >>>>>>>>> Martell >>>>>>>>> >>>>>>>>> >>>>>>>>> Context >>>>>>>>> =========>>>>>>>>> >>>>>>>>> Awhile back I talked to various llvm'ers about getting mingw-w64 >>>>>>>>> support for lld. >>>>>>>>> There were a few issues raised but the main issue was that >>>>>>>>> mingw-w64 should try best to comply with the PECOFF spec, adding support >>>>>>>>> for custom sections and various binutils/mingw hacks would impact the >>>>>>>>> performance of the COFF linker and in general is not something that lld >>>>>>>>> should support. >>>>>>>>> >>>>>>>> >>>>>>>> It's not a performance issue but a code maintenance issue. The >>>>>>>> initial patches to support mingw was trying to add a new linker driver and >>>>>>>> a different linkin semantics to the COFF linker which seemed too >>>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU >>>>>>>> command line arguments to MSVC linker arguments. Didn't it work? >>>>>>>> >>>>>>>> >>>>>>>>> Motivation >>>>>>>>> =========>>>>>>>>> >>>>>>>>> The main motivation was because dlltool and ld did not comply with >>>>>>>>> PECOFF Spec. >>>>>>>>> It has some custom formatting and uses the assembler for some >>>>>>>>> reason to generate import libraries, it did not use the short import >>>>>>>>> library format at all. >>>>>>>>> >>>>>>>>> There has been many work arounds for the problem this creates such >>>>>>>>> as the creation of the `reimp` tool. Which imports MSVC built libs creates >>>>>>>>> a def and uses dlltool so that the binutils linker can use it. >>>>>>>>> >>>>>>>>> We should just be using the short import format and the linker >>>>>>>>> should support that. >>>>>>>>> Thus llvm-dlltool was born. >>>>>>>>> >>>>>>>>> >>>>>>>>> Difference from lib >>>>>>>>> ==================>>>>>>>>> >>>>>>>>> Using PE COFF spec (section 8, Import Library Format) should be >>>>>>>>> self explanatory. >>>>>>>>> lib.exe is able to accept def files and create libraries using >>>>>>>>> this format. >>>>>>>>> >>>>>>>>> example >>>>>>>>> >>>>>>>>> `LIBRARY "user32.dll" >>>>>>>>> EXPORTS >>>>>>>>> MessageBoxA` >>>>>>>>> >>>>>>>>> LIB.exe can create a user32.lib with the function MessageBoxA from >>>>>>>>> the above definition. >>>>>>>>> >>>>>>>>> Mingw-w64 is different MSVC in that we need to compile the runtime. >>>>>>>>> MS provide us with their crt prebuilt so lib.exe doesn't have >>>>>>>>> support for external function aliasing. >>>>>>>>> We often use aliases for posix naming reasons as well as avoid >>>>>>>>> using the MS version of a function. >>>>>>>>> >>>>>>>>> example >>>>>>>>> >>>>>>>>> `LIBRARY "user32.dll" >>>>>>>>> EXPORTS >>>>>>>>> MessageBoxA >>>>>>>>> MessageBoxW==MessageBoxA` >>>>>>>>> >>>>>>>>> LIB.exe doesn't not have support for this but dlltool does. >>>>>>>>> The best fit for this within the spec is PE COFF spec (Aux Format >>>>>>>>> 3: Weak Externals) >>>>>>>>> >>>>>>>>> Mingw-w64 also uses lib prefix and .a file extensions for the >>>>>>>>> library name so the driver should be different. The above example would be >>>>>>>>> libuser32.a, for when shared and static versions of the same library exist >>>>>>>>> there is the .dll.a variant. >>>>>>>>> >>>>>>>>> >>>>>>>>> Issues >>>>>>>>> ======>>>>>>>>> >>>>>>>>> Like more of the gnu suite dlltool was not designed in one build >>>>>>>>> multi target manner. >>>>>>>>> We would have to introduce custom arguments to specify the target, >>>>>>>>> "i686", "x86_64", "thumb2pe" etc. >>>>>>>>> This will most likely break some level of backwards compatibility >>>>>>>>> with the binutils version. >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> LLVM Developers mailing list >>>>>>>>> llvm-dev at lists.llvm.org >>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170213/c45a1dda/attachment-0001.html>
Peter Collingbourne via llvm-dev
2017-Feb-14 01:26 UTC
[llvm-dev] RFC: A new llvm-dlltool driver and llvm-lib driver improvements
On Mon, Feb 13, 2017 at 5:20 PM, Rui Ueyama via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On Mon, Feb 13, 2017 at 4:46 PM, Martell Malone <martellmalone at gmail.com> > wrote: > >> No, I meant an even thinner wrapper which textually translates arguments. >>> For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o >>> foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything >>> with Config object nor LinkerDriver::run and have absolutely zero knowledge >>> on the internals of LLD. >> >> Ohh okay I misunderstood. >> >> (Why I'm asking this is because I want to sure that there's no difference >>> between the regular Windows linker and mingw. If all differences are >>> superficial, the command line translator should just work. If not, there's >>> some semantic difference.) >> >> The libraries can be used with MSVC link.exe directly assuming the >> correct arguments are passed to the linker. >> I tested this after creating a working llvm-dlltool. >> The only change I had to make to support this was alter ming-w64 crt to >> change all references from __image_base__ to _ImageBase to support that >> > > You may be able to define __ImageBase as a weak external symbol to > __image_base__. Then, if __ImageBase is not defined, all references against > __ImageBase will be resolved using __image_base__. >Or you could have your wrapper driver pass "/alternatename:__image_base__=__ImageBase". Peter> > >> That should be fine. >> >> Great >> >> On Mon, Feb 13, 2017 at 9:34 PM, Rui Ueyama <ruiu at google.com> wrote: >> >>> On Mon, Feb 13, 2017 at 1:16 PM, Martell Malone <martellmalone at gmail.com >>> > wrote: >>> >>>> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process >>>>> wrapper, meaning that you could add a main function for mingw which >>>>> translates all arguments into the MSVC style and then invoke the COFF >>>>> linker's main function. >>>> >>>> That should work, if I am understanding this correctly I can create an >>>> argument parser (probably partially based on the ELF one) then I can set >>>> the data in the COFF Config object and call COFF LinkerDriver::run etc >>>> directly from that? >>>> >>> >>> No, I meant an even thinner wrapper which textually translates >>> arguments. For example, the wrapper would translates "/out:foo.exe foo.obj" >>> to "-o foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do >>> anything with Config object nor LinkerDriver::run and have absolutely zero >>> knowledge on the internals of LLD. >>> >>> (Why I'm asking this is because I want to sure that there's no >>> difference between the regular Windows linker and mingw. If all differences >>> are superficial, the command line translator should just work. If not, >>> there's some semantic difference.) >>> >>> This proposal is for generally moving code from lld into llvm that could >>>> be part of lib.exe / dlltool, what are your thoughts here? >>>> >>> >>> That should be fine. >>> >>> >>>> On Mon, Feb 13, 2017 at 8:57 PM, Rui Ueyama <ruiu at google.com> wrote: >>>> >>>>> On Mon, Feb 13, 2017 at 12:52 PM, Martell Malone < >>>>> martellmalone at gmail.com> wrote: >>>>> >>>>>> Also you need to make a change to LLD/COFF to accept GNU command >>>>>>> arguments, right? (Looks like you already have that patch locally.) >>>>>> >>>>>> Yes >>>>>> >>>>> >>>>> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process >>>>> wrapper, meaning that you could add a main function for mingw which >>>>> translates all arguments into the MSVC style and then invoke the COFF >>>>> linker's main function. >>>>> >>>>> >>>>>> >>>>>> My patch to hack lld into accepting some very basic gnu front end >>>>>>> arguments was enough to get all the above working which was enough to >>>>>>> develop further. >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Feb 13, 2017 at 8:41 PM, Rui Ueyama <ruiu at google.com> wrote: >>>>>> >>>>>>> On Mon, Feb 13, 2017 at 12:33 PM, Martell Malone < >>>>>>> martellmalone at gmail.com> wrote: >>>>>>> >>>>>>>> Hey Rui, >>>>>>>> >>>>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw >>>>>>>>> support. I don't think dlltool is the last missing piece. What do you need >>>>>>>>> to do other than that to fully support mingw using LLVM toolchain? >>>>>>>> >>>>>>>> Other then changing `lib/MC/WinCOFFStreamer.cpp` to not use >>>>>>>> -aligncomm within the EmitCommonSymbol function and a single patch for >>>>>>>> mingw-w64 itself to pre-populate it's .ctors and .dtors list, so >>>>>>>> llvm-dlltool is infact the only missing part really. >>>>>>>> >>>>>>> >>>>>>> Also you need to make a change to LLD/COFF to accept GNU command >>>>>>> arguments, right? (Looks like you already have that patch locally.) >>>>>>> >>>>>>> >>>>>>>> This gives us a fully working clang based mingw-w64 C compiler. >>>>>>>> >>>>>>>> C++ and exception handling is a different story. >>>>>>>> >>>>>>>> libc++ is somewhat working with the following test results >>>>>>>> >>>>>>>> Expected Passes : 2188 >>>>>>>> Expected Failures : 44 >>>>>>>> Unsupported Tests : 588 >>>>>>>> Unexpected Failures: 2816 >>>>>>>> >>>>>>>> I was able to build it with exceptions disabled and I actually >>>>>>>> managed to bootstrap llvm and clang itself. >>>>>>>> It didn't run very well though as you can imagine based on the >>>>>>>> above tests. >>>>>>>> >>>>>>>> It's not a performance issue but a code maintenance issue. The >>>>>>>>> initial patches to support mingw was trying to add a new linker driver and >>>>>>>>> a different linkin semantics to the COFF linker which seemed too >>>>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU >>>>>>>>> command line arguments to MSVC linker arguments. Didn't it work? >>>>>>>> >>>>>>>> There were just so many differences between link and ld I never >>>>>>>> followed down that path. I pushed forward with the short import library >>>>>>>> support based on your later suggestions. My patch to hack lld into >>>>>>>> accepting some very basic gnu front end arguments was enough to get all the >>>>>>>> above working which was enough to develop further. >>>>>>>> >>>>>>>> Best, >>>>>>>> Martell >>>>>>>> >>>>>>>> On Mon, Feb 13, 2017 at 8:08 PM, Rui Ueyama <ruiu at google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Martell, >>>>>>>>> >>>>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw >>>>>>>>> support. I don't think dlltool is the last missing piece. What do you need >>>>>>>>> to do other than that to fully support mingw using LLVM toolchain? >>>>>>>>> >>>>>>>>> On Mon, Feb 13, 2017 at 8:56 AM, Martell Malone via llvm-dev < >>>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>>> >>>>>>>>>> Hey llvm'ers, >>>>>>>>>> >>>>>>>>>> I have been working on a dlltool replacement for llvm. >>>>>>>>>> Here is my initial differential https://reviews.llvm.org/D29892 >>>>>>>>>> It is based on some functionality that already exists in lld. >>>>>>>>>> I added functionality to support, PE COFF Weak Externals and of >>>>>>>>>> course a front end to actually use it. >>>>>>>>>> I believe the work here can also be used for llvm-lib and lessen >>>>>>>>>> the load on lld. >>>>>>>>>> I would like some comments about how this could be be structured >>>>>>>>>> to live in llvm with a shared code base across lib ar and dlltool. >>>>>>>>>> I also have a section below called "Difference from lib" which is >>>>>>>>>> somewhat of a rationale for the tool. >>>>>>>>>> >>>>>>>>>> Many Thanks, >>>>>>>>>> Martell >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Context >>>>>>>>>> =========>>>>>>>>>> >>>>>>>>>> Awhile back I talked to various llvm'ers about getting mingw-w64 >>>>>>>>>> support for lld. >>>>>>>>>> There were a few issues raised but the main issue was that >>>>>>>>>> mingw-w64 should try best to comply with the PECOFF spec, adding support >>>>>>>>>> for custom sections and various binutils/mingw hacks would impact the >>>>>>>>>> performance of the COFF linker and in general is not something that lld >>>>>>>>>> should support. >>>>>>>>>> >>>>>>>>> >>>>>>>>> It's not a performance issue but a code maintenance issue. The >>>>>>>>> initial patches to support mingw was trying to add a new linker driver and >>>>>>>>> a different linkin semantics to the COFF linker which seemed too >>>>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU >>>>>>>>> command line arguments to MSVC linker arguments. Didn't it work? >>>>>>>>> >>>>>>>>> >>>>>>>>>> Motivation >>>>>>>>>> =========>>>>>>>>>> >>>>>>>>>> The main motivation was because dlltool and ld did not comply >>>>>>>>>> with PECOFF Spec. >>>>>>>>>> It has some custom formatting and uses the assembler for some >>>>>>>>>> reason to generate import libraries, it did not use the short import >>>>>>>>>> library format at all. >>>>>>>>>> >>>>>>>>>> There has been many work arounds for the problem this creates >>>>>>>>>> such as the creation of the `reimp` tool. Which imports MSVC built libs >>>>>>>>>> creates a def and uses dlltool so that the binutils linker can use it. >>>>>>>>>> >>>>>>>>>> We should just be using the short import format and the linker >>>>>>>>>> should support that. >>>>>>>>>> Thus llvm-dlltool was born. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Difference from lib >>>>>>>>>> ==================>>>>>>>>>> >>>>>>>>>> Using PE COFF spec (section 8, Import Library Format) should be >>>>>>>>>> self explanatory. >>>>>>>>>> lib.exe is able to accept def files and create libraries using >>>>>>>>>> this format. >>>>>>>>>> >>>>>>>>>> example >>>>>>>>>> >>>>>>>>>> `LIBRARY "user32.dll" >>>>>>>>>> EXPORTS >>>>>>>>>> MessageBoxA` >>>>>>>>>> >>>>>>>>>> LIB.exe can create a user32.lib with the function MessageBoxA >>>>>>>>>> from the above definition. >>>>>>>>>> >>>>>>>>>> Mingw-w64 is different MSVC in that we need to compile the >>>>>>>>>> runtime. >>>>>>>>>> MS provide us with their crt prebuilt so lib.exe doesn't have >>>>>>>>>> support for external function aliasing. >>>>>>>>>> We often use aliases for posix naming reasons as well as avoid >>>>>>>>>> using the MS version of a function. >>>>>>>>>> >>>>>>>>>> example >>>>>>>>>> >>>>>>>>>> `LIBRARY "user32.dll" >>>>>>>>>> EXPORTS >>>>>>>>>> MessageBoxA >>>>>>>>>> MessageBoxW==MessageBoxA` >>>>>>>>>> >>>>>>>>>> LIB.exe doesn't not have support for this but dlltool does. >>>>>>>>>> The best fit for this within the spec is PE COFF spec (Aux >>>>>>>>>> Format 3: Weak Externals) >>>>>>>>>> >>>>>>>>>> Mingw-w64 also uses lib prefix and .a file extensions for the >>>>>>>>>> library name so the driver should be different. The above example would be >>>>>>>>>> libuser32.a, for when shared and static versions of the same library exist >>>>>>>>>> there is the .dll.a variant. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Issues >>>>>>>>>> ======>>>>>>>>>> >>>>>>>>>> Like more of the gnu suite dlltool was not designed in one build >>>>>>>>>> multi target manner. >>>>>>>>>> We would have to introduce custom arguments to specify the >>>>>>>>>> target, "i686", "x86_64", "thumb2pe" etc. >>>>>>>>>> This will most likely break some level of backwards compatibility >>>>>>>>>> with the binutils version. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> LLVM Developers mailing list >>>>>>>>>> llvm-dev at lists.llvm.org >>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-- -- Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170213/70e0cfb0/attachment.html>
Possibly Parallel Threads
- RFC: A new llvm-dlltool driver and llvm-lib driver improvements
- RFC: A new llvm-dlltool driver and llvm-lib driver improvements
- RFC: A new llvm-dlltool driver and llvm-lib driver improvements
- RFC: A new llvm-dlltool driver and llvm-lib driver improvements
- RFC: A new llvm-dlltool driver and llvm-lib driver improvements