llvm dev - Mar 2019 - Missing data on PDB's generated by lld

No, we use msvc cl.exe as a compiler generating C7 debug information
(here is more or less our compiler options regarding
optimization/debugability):

/Z7 /GL- /bigobj /Zo /utf-8 /arch:AVX /std:c++14 /Zc:inline
/Zc:ternary /Zc:rvalueCast /Zc:strictStrings /O2 /MD

In this list the only weird one is /bigobj, my main fear is that we
might get to the limit of sections on an obj and then by adding the
.debug$H with llvm-objcopy we might get over the limit (I didn't see
any code on the obj coff writer to deal with that).


For the linker I was using:

/FUNCTIONPADMIN:6 /MACHINE:X64 /DEBUG:GHASH -time /INCREMENTAL:NO

But I'm going to be adding: /OPT:NOREF /OPT:NOICF because maybe the
wrong callstacks might be related to ICF.

I will try to get a simple test case to show this erros, but its
difficult, debugging mostly works, its just on some complicated cases
that it stops working, but we might be able to find a small repro
case. This email was to get to know how are you all dealing with these
on big projects.

On Mon, Mar 18, 2019 at 9:36 PM Zachary Turner <zturner at google.com>
wrote:
>
> You mentioned that the binaries are generated by LLD, I assume the object
files are also generated by clang-cl?
>
> Also, is this at -O0 or something higher?
>
> A reproduction would really help, as it would allow us to investigate this
on our end.  Obviously your code is proprietary, but it might be possible to
reconstruct a similar enough call-stack and type structure to reproduce it in a
simpler program.
>
> On Mon, Mar 18, 2019 at 12:52 PM Leonardo Santagada via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>
>> Hi,
>>
>> We are starting to test binaries generated by lld on windows and we
>> notices that sometimes the visual studio debugger can't see the
>> content of variables or gets the call stack wrong for deeply nested or
>> complex types. Did anyone else have the same problems, or any way to
>> try to figure out what is missing? I tried llvm-pdbutil dump -symbols
>> but there is a lot of small diferences on pdb's generated by taking
>> msvc C7 debug information and linking with lld and msvc generating
>> pdbs and linking with link.exe so its hard to see any big parts
>> missing.
>>
>> --
>>
>> Leonardo Santagada
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


-- 

Leonardo Santagada

On Mon, 18 Mar 2019, Leonardo Santagada via llvm-dev wrote:
> In this list the only weird one is /bigobj, my main fear is that we
> might get to the limit of sections on an obj and then by adding the
> .debug$H with llvm-objcopy we might get over the limit (I didn't see
> any code on the obj coff writer to deal with that).
The COFF backend in llvm-objcopy is supposed to handle this - it picks 
between normal and big objs based on the number of sections when writing 
it out. The object type on input shouldn't matter:

https://github.com/llvm-project/llvm/blob/master/tools/llvm-objcopy/COFF/Writer.cpp#L407

This is at least the intent (for which there's at least some unit tests), 
but there could of course be cases I didn't think of.

// Martin

Thanks, that is very reassuring, don't know how I missed this code. I
will probably remove the /bigobj... although cl.exe IIRC just throws
and error when you go over the limit, and we are probably very close
with obj files on the tens of megabytes.

On Mon, Mar 18, 2019 at 11:46 PM Martin Storsjö <martin at martin.st>
wrote:
>
> On Mon, 18 Mar 2019, Leonardo Santagada via llvm-dev wrote:
>
> > In this list the only weird one is /bigobj, my main fear is that we
> > might get to the limit of sections on an obj and then by adding the
> > .debug$H with llvm-objcopy we might get over the limit (I didn't
see
> > any code on the obj coff writer to deal with that).
>
> The COFF backend in llvm-objcopy is supposed to handle this - it picks
> between normal and big objs based on the number of sections when writing
> it out. The object type on input shouldn't matter:
>
>
https://github.com/llvm-project/llvm/blob/master/tools/llvm-objcopy/COFF/Writer.cpp#L407
>
> This is at least the intent (for which there's at least some unit
tests),
> but there could of course be cases I didn't think of.
>
> // Martin
>

-- 

Leonardo Santagada