llvm dev - Jan 2018 - [lldb-dev] Trying out lld to link windows binaries (using msvc as a compiler)

-lldb-dev, +llvm-dev

lldb-dev is specifically for the llvm debugger, and llvm-dev is for
everything else (including lld)

Did you compile the object files with clang and use -mllvm
-emit-codeview-ghash-section?

It sounds like probably not.  If you don’t do that /DEBUG:GHASH will indeed
be much slower.  I wonder if we should emit a linker diagnostic in this case

On Fri, Jan 19, 2018 at 12:09 PM Leonardo Santagada via lldb-dev <
lldb-dev at lists.llvm.org> wrote:
> Hi all,
>
> At work I have been experimenting with linking with lld, I can give all
> sorts of information about the performance so far if someone wants to hear
> it, but for a large binary (around 100mb and 800mb of symbols) it takes
> considerably more memory and time than the vs 2015 and 2017 linker.
>
> What I've been playing around is the idea of creating a tool that
generate
> those DEBUG:GHASH sections on .obj files already created, I saw code for
> reading them in lld and started looking at the codeview part of clang, is
> there any documentation around that, and would anyone else have interest in
> such a tool? I wish we could just jump shit to clang but right now that is
> not a possibility (It mostly revolves around windows.h and the copy of it
> we mantain).
>
>
> ps: I have two patches that I needed to get it to finish linking I can
> submit them monday.
>
> --
>
> Leonardo Santagada
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180119/2a69e05a/attachment.html>

Hi,

No I didn't, I used cl.exe from the visual studio toolchain. What I'm
proposing is a tool for processing .obj files in COFF format, reading them
and generating the GHASH part.

To make our build faster we use hundreds of unity build files (.cpp's with
a lot of other .cpp's in them aka munch files) but still have a lot of
single .cpp's as well (in total something like 3.4k .obj files).

ps: sorry for sending to the wrong list, I was reading about llvm mailing
lists and jumped when I saw what I thought was a lld exclusive list.

On Fri, Jan 19, 2018 at 9:17 PM, Zachary Turner <zturner at google.com>
wrote:
> -lldb-dev, +llvm-dev
>
> lldb-dev is specifically for the llvm debugger, and llvm-dev is for
> everything else (including lld)
>
> Did you compile the object files with clang and use -mllvm
> -emit-codeview-ghash-section?
>
> It sounds like probably not.  If you don’t do that /DEBUG:GHASH will
> indeed be much slower.  I wonder if we should emit a linker diagnostic in
> this case
>
> On Fri, Jan 19, 2018 at 12:09 PM Leonardo Santagada via lldb-dev <
> lldb-dev at lists.llvm.org> wrote:
>
>> Hi all,
>>
>> At work I have been experimenting with linking with lld, I can give all
>> sorts of information about the performance so far if someone wants to
hear
>> it, but for a large binary (around 100mb and 800mb of symbols) it takes
>> considerably more memory and time than the vs 2015 and 2017 linker.
>>
>> What I've been playing around is the idea of creating a tool that
>> generate those DEBUG:GHASH sections on .obj files already created, I
saw
>> code for reading them in lld and started looking at the codeview part
of
>> clang, is there any documentation around that, and would anyone else
have
>> interest in such a tool? I wish we could just jump shit to clang but
right
>> now that is not a possibility (It mostly revolves around windows.h and
the
>> copy of it we mantain).
>>
>>
>> ps: I have two patches that I needed to get it to finish linking I can
>> submit them monday.
>>
>> --
>>
>> Leonardo Santagada
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>

-- 

Leonardo Santagada
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180119/fb468c9e/attachment.html>

Zachary,

Do you generally recommend using -mllvm -emit-codeview-ghash-section and
/DEBUG:GHASH when linking for Windows?

On Fri, Jan 19, 2018 at 3:17 PM, Zachary Turner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> -lldb-dev, +llvm-dev
>
> lldb-dev is specifically for the llvm debugger, and llvm-dev is for
> everything else (including lld)
>
> Did you compile the object files with clang and use -mllvm
> -emit-codeview-ghash-section?
>
> It sounds like probably not.  If you don’t do that /DEBUG:GHASH will
> indeed be much slower.  I wonder if we should emit a linker diagnostic in
> this case
>
> On Fri, Jan 19, 2018 at 12:09 PM Leonardo Santagada via lldb-dev <
> lldb-dev at lists.llvm.org> wrote:
>
>> Hi all,
>>
>> At work I have been experimenting with linking with lld, I can give all
>> sorts of information about the performance so far if someone wants to
hear
>> it, but for a large binary (around 100mb and 800mb of symbols) it takes
>> considerably more memory and time than the vs 2015 and 2017 linker.
>>
>> What I've been playing around is the idea of creating a tool that
>> generate those DEBUG:GHASH sections on .obj files already created, I
saw
>> code for reading them in lld and started looking at the codeview part
of
>> clang, is there any documentation around that, and would anyone else
have
>> interest in such a tool? I wish we could just jump shit to clang but
right
>> now that is not a possibility (It mostly revolves around windows.h and
the
>> copy of it we mantain).
>>
>>
>> ps: I have two patches that I needed to get it to finish linking I can
>> submit them monday.
>>
>> --
>>
>> Leonardo Santagada
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180119/c7ba93c3/attachment.html>

On Fri, Jan 19, 2018 at 12:29 PM Leonardo Santagada <santagada at
gmail.com>
wrote:
> Hi,
>
> No I didn't, I used cl.exe from the visual studio toolchain. What
I'm
> proposing is a tool for processing .obj files in COFF format, reading them
> and generating the GHASH part.
>
> To make our build faster we use hundreds of unity build files (.cpp's
with
> a lot of other .cpp's in them aka munch files) but still have a lot of
> single .cpp's as well (in total something like 3.4k .obj files).
>
> ps: sorry for sending to the wrong list, I was reading about llvm mailing
> lists and jumped when I saw what I thought was a lld exclusive list.
>
A tool like this would be useful, yes.  We've talked about it internally as
well and agreed it would be useful, we just haven't prioritized it.  If
you're interested in submitting a patch along those lines though, I think
it would be a good addition.

I'm not sure what the best place for it would be.  llvm-readobj and
llvm-objdump seem like obvious choices, but they are intended to be
read-only, so perhaps they wouldn't be a good fit.

llvm-pdbutil is kind of a hodgepodge of everything else related to PDBs and
symbols, so I wouldn't be opposed to making a new subcommand there called
"ghash" or something that could process an object file and output a
new
object file with a .debug$H section.

A third option would be to make a new tool for it.

I don't htink it would be that hard to write.  If you're interested in
trying to make a patch for this, I can offer some guidance on where to look
in the code.  Otherwise it's something that we'll probably get to,
I'm just
not sure when.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180119/146e560a/attachment.html>

On Fri, Jan 19, 2018 at 12:30 PM Andrew Kelley <superjoe30 at gmail.com>
wrote:
> Zachary,
>
> Do you generally recommend using -mllvm -emit-codeview-ghash-section and
> /DEBUG:GHASH when linking for Windows?
>
I definitely want you to try it out and let me know how it goes :)   It's
behind the undocumented -emit-codeview-ghash-section for now until we have
some more data indicating it's safe to make default.  I'd like to
eventually make it the default, so that you won't have to specify anything
on the compiler side other than /Z[i|7].

So far I am not aware of any downsides of using -mllvm
-emit-codeview-ghash-section and /DEBUG:GHASH (aside from a small increase
in object file size).  But if you find any let me know.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180119/9e094685/attachment.html>