llvm dev - Nov 2013 - [LLVMdev] Adding line table debug information to LLVM on Windows

Hi David, Eric, LLVM devs,

You've probably heard about AddressSanitizer (ASan) and other
sanitizers based on LLVM. One of the things that makes ASan
not as awesome on Windows as it is on Linux
is the symbolization of the stacks.

Currently, ASan runtime on Windows uses
CaptureStackBackTrace/SymFromAddr/SymGetLineFromAddr64
to unwind and symbolize stacks.  This works like a charm
in-process for stack frames built with CL, but yields
"function+0x0ff537" for frames built with Clang.

I came up with a prototype which emits "old-style debug info" COFF
sections that are sufficient to get function name / filename /
linenumber information.  That's pretty much everything that's required
for ASan to work beautifully in terms of the completeness of error
reports.

Attached is a prototype patch which I've tried on some simple tests,
including some more complex ones with weird #line constructions.
It also works just great on ASan/Win tests without any link/run-time
warnings (I had a bunch of those during development, so I can tell it
works rather than fails silently).

I didn't have time to work on threading the command-line flags into
the AsmPrinter yet, so currently it just replaces DWARF entirely.
Of course, this should be fixed before this lands into trunk.
Currently, one can try this patch by using "clang-cl -Xclang -g ...".
Eventually we should have some dedicated flag for clang-cl.

Can you please take a look at the patch and suggest a good path forward?

I'm very unfamiliar with LLVM CodeGen/MC, so I'm pretty sure I made a
few weird things in the code...  I've also put a few TODOs with
questions and suggestions.

Some general questions:
1) Threading flags from the driver down to CodeGen.
  How do we do that? Should we support all 4 combinations
  of no-info/DWARF/CVLT/both?
  How about "-Zmlt" as the clang-cl flag name? ("minimal line
tables")

2) Am I right that DWARF is pretty much the only debug info format
  supported by LLVM/AsmPrinter right now?  Do we want to take
  an effort to come up with a generic debuginfogenerator interface
  to share between DwarfDebug and WinCodeViewLineTables?
  Then AsmPrinter should just hold a SmallVector<DebugInfoEmitter*>
  rather than a pair of DD/DE pointers.

3) How would you suggest to write WinCodeViewLineTables tests
  given that dumpbin is not available everywhere except Windows?
  // Yeah, I should have looked at the DwarfDebug LIT tests and
  // written some; but the prototype development went faster
  // than I expected...

Can you suggest ways to split this patch so it's easier
to review part-by-part before this hits trunk?

Thanks!
--
Timur Iskhodzhanov,
Google
-------------- next part --------------
A non-text attachment was scrubbed...
Name: codeview_linetables.patch
Type: application/octet-stream
Size: 36333 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/0eea4311/attachment.obj>

2013/11/14 Timur Iskhodzhanov <timurrrr at
google.com>:
> Hi David, Eric, LLVM devs,
>
> You've probably heard about AddressSanitizer (ASan) and other
> sanitizers based on LLVM. One of the things that makes ASan
> not as awesome on Windows as it is on Linux
> is the symbolization of the stacks.
>
> Currently, ASan runtime on Windows uses
> CaptureStackBackTrace/SymFromAddr/SymGetLineFromAddr64
> to unwind and symbolize stacks.  This works like a charm
> in-process for stack frames built with CL, but yields
> "function+0x0ff537" for frames built with Clang.
>
> I came up with a prototype which emits "old-style debug info"
COFF
> sections that are sufficient to get function name / filename /
> linenumber information.  That's pretty much everything that's
required
> for ASan to work beautifully in terms of the completeness of error
> reports.
>
> Attached is a prototype patch which I've tried on some simple tests,
> including some more complex ones with weird #line constructions.
> It also works just great on ASan/Win tests without any link/run-time
> warnings (I had a bunch of those during development, so I can tell it
> works rather than fails silently).
>
> I didn't have time to work on threading the command-line flags into
> the AsmPrinter yet, so currently it just replaces DWARF entirely.
> Of course, this should be fixed before this lands into trunk.
> Currently, one can try this patch by using "clang-cl -Xclang -g
...".
> Eventually we should have some dedicated flag for clang-cl.
>
> Can you please take a look at the patch and suggest a good path forward?
>
> I'm very unfamiliar with LLVM CodeGen/MC, so I'm pretty sure I made
a
> few weird things in the code...  I've also put a few TODOs with
> questions and suggestions.
>
> Some general questions:
> 1) Threading flags from the driver down to CodeGen.
>   How do we do that? Should we support all 4 combinations
>   of no-info/DWARF/CVLT/both?
>   How about "-Zmlt" as the clang-cl flag name? ("minimal
line tables")
>
> 2) Am I right that DWARF is pretty much the only debug info format
>   supported by LLVM/AsmPrinter right now?  Do we want to take
>   an effort to come up with a generic debuginfogenerator interface
>   to share between DwarfDebug and WinCodeViewLineTables?
>   Then AsmPrinter should just hold a SmallVector<DebugInfoEmitter*>
>   rather than a pair of DD/DE pointers.
>
> 3) How would you suggest to write WinCodeViewLineTables tests
>   given that dumpbin is not available everywhere except Windows?
>   // Yeah, I should have looked at the DwarfDebug LIT tests and
>   // written some; but the prototype development went faster
>   // than I expected...
I found the MCAsmStreamer being used by llc which gives a decent text format.
I wrote a simple x86+x86_64 .ll test and FileCheck expectations for
the llc asm output.
Is this the right approach to write tests?
If so, I'll convert my remaining C program test cases into such an
.ll+llc tests.
> Can you suggest ways to split this patch so it's easier
> to review part-by-part before this hits trunk?
Attached is an updated patch with a new test and a few minor things improved.
I also removed the "TODO: test on X64" as I did try it on x64 and no
changes were required.

Looking forward to your feedback!
> Thanks!
> --
> Timur Iskhodzhanov,
> Google
-------------- next part --------------
A non-text attachment was scrubbed...
Name: codeview_linetables.patch
Type: application/octet-stream
Size: 43569 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131115/31b4cee0/attachment.obj>

Hi Timur,

There's also a pending patch adding CodeView support in Phab:
http://llvm-reviews.chandlerc.com/D165

Does your patch provide just a subset of the CodeView debug info provided
in the other patch?

Looking at the patch, I think the approach the other patch took of
abstracting the emission of debug information is a bit cleaner and it will
probably make life easier when adding more debug formats in the future.





On Fri, Nov 15, 2013 at 4:39 PM, Timur Iskhodzhanov <timurrrr at
google.com>wrote:
> 2013/11/14 Timur Iskhodzhanov <timurrrr at google.com>:
> > Hi David, Eric, LLVM devs,
> >
> > You've probably heard about AddressSanitizer (ASan) and other
> > sanitizers based on LLVM. One of the things that makes ASan
> > not as awesome on Windows as it is on Linux
> > is the symbolization of the stacks.
> >
> > Currently, ASan runtime on Windows uses
> > CaptureStackBackTrace/SymFromAddr/SymGetLineFromAddr64
> > to unwind and symbolize stacks.  This works like a charm
> > in-process for stack frames built with CL, but yields
> > "function+0x0ff537" for frames built with Clang.
> >
> > I came up with a prototype which emits "old-style debug
info" COFF
> > sections that are sufficient to get function name / filename /
> > linenumber information.  That's pretty much everything that's
required
> > for ASan to work beautifully in terms of the completeness of error
> > reports.
> >
> > Attached is a prototype patch which I've tried on some simple
tests,
> > including some more complex ones with weird #line constructions.
> > It also works just great on ASan/Win tests without any link/run-time
> > warnings (I had a bunch of those during development, so I can tell it
> > works rather than fails silently).
> >
> > I didn't have time to work on threading the command-line flags
into
> > the AsmPrinter yet, so currently it just replaces DWARF entirely.
> > Of course, this should be fixed before this lands into trunk.
> > Currently, one can try this patch by using "clang-cl -Xclang -g
...".
> > Eventually we should have some dedicated flag for clang-cl.
> >
> > Can you please take a look at the patch and suggest a good path
forward?
> >
> > I'm very unfamiliar with LLVM CodeGen/MC, so I'm pretty sure I
made a
> > few weird things in the code...  I've also put a few TODOs with
> > questions and suggestions.
> >
> > Some general questions:
> > 1) Threading flags from the driver down to CodeGen.
> >   How do we do that? Should we support all 4 combinations
> >   of no-info/DWARF/CVLT/both?
> >   How about "-Zmlt" as the clang-cl flag name?
("minimal line tables")
> >
> > 2) Am I right that DWARF is pretty much the only debug info format
> >   supported by LLVM/AsmPrinter right now?  Do we want to take
> >   an effort to come up with a generic debuginfogenerator interface
> >   to share between DwarfDebug and WinCodeViewLineTables?
> >   Then AsmPrinter should just hold a
SmallVector<DebugInfoEmitter*>
> >   rather than a pair of DD/DE pointers.
> >
> > 3) How would you suggest to write WinCodeViewLineTables tests
> >   given that dumpbin is not available everywhere except Windows?
> >   // Yeah, I should have looked at the DwarfDebug LIT tests and
> >   // written some; but the prototype development went faster
> >   // than I expected...
>
> I found the MCAsmStreamer being used by llc which gives a decent text
> format.
> I wrote a simple x86+x86_64 .ll test and FileCheck expectations for
> the llc asm output.
> Is this the right approach to write tests?
> If so, I'll convert my remaining C program test cases into such an
> .ll+llc tests.
>
> > Can you suggest ways to split this patch so it's easier
> > to review part-by-part before this hits trunk?
>
> Attached is an updated patch with a new test and a few minor things
> improved.
> I also removed the "TODO: test on X64" as I did try it on x64 and
no
> changes were required.
>
> Looking forward to your feedback!
>
> > Thanks!
> > --
> > Timur Iskhodzhanov,
> > Google
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>

-- 
João Matos
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131115/b0377640/attachment.html>