llvm dev - Nov 2015 - [cfe-dev] RFC: Supporting macros in LLVM debug info

On Mon, Nov 9, 2015 at 4:00 AM, Aboud, Amjad <amjad.aboud at intel.com>
wrote:
> I found a way to skip representing macros in AST and create them directly
> in CGDebugInfo through PPCallbacks during preprocessing.
>
> To do that, I needed to extend ASTConsumer interface with this extra
> method:
>
>
>
>   /// If the consumer is interested in notifications from Preprocessor,
>
>   /// for example: notifications on macro definitions, etc., it should
> return
>
>   /// a pointer to a PPCallbacks here.
>
>   /// The caller takes ownership on the returned pointer.
>
>   virtual PPCallbacks *CreatePreprocessorCallbacks() { return nullptr; }
>
>
>
> Then the ParseAST can use it to add these preprocessor callbacks, which
> are needed by the AST consumer, to the preprocessor:
>
>
>
>   S.getPreprocessor().addPPCallbacks(
>
>       std::unique_ptr<PPCallbacks
> >(Consumer->CreatePreprocessorCallbacks()));
>
(CreatePreprocessorCallbacks, if that's the path we take, should return a
unique_ptr itself rather than returning a raw ownership-passing pointer,
but that's a minor API detail)

>
>
> With this, approach the change in clang to support macros is very small.
>
>
>
> Do you agree to this approach?
>
Richard - what do you reckon's the right hook/path to get preprocessor info
through to codegen (& CGDebugInfo in particular). Would a general purpose
hook in the ASTConsumer be appropriate/useful?

>
>
> Thanks,
>
> Amjad
>
>
>
> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of
*Aboud,
> Amjad via llvm-dev
> *Sent:* Thursday, November 05, 2015 16:56
> *To:* David Blaikie
>
> *Cc:* llvm-dev at lists.llvm.org
> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>
>
>
> > Right - I was wondering if CGDebugInfo already implemented PPCallbacks
> or was otherwise being notified of PPCallback related things, possibly
> through a layer or two of indirection.
>
>
>
> I checked the approach of skipping representing macros in AST, and
> communicate them directly from Parser to CGDebugInfo.
>
> However, I could not find a way to initialize this communication.
>
> The only interface available through Parser is either Sema (to create an
> AST) or ASTConsumer. While the CGDebugInfo is only available in the
> CodeGenModule, which is accessible from BackendConsumer, that implements
> ASTConsumer.
>
>
>
> David, skipping the AST will save a lot of code, but I need help figuring
> out how to communicate with the CGDebugInfo.
>
>
>
> Thanks,
>
> Amjad
>
>
>
> *From:* David Blaikie [mailto:dblaikie at gmail.com <dblaikie at
gmail.com>]
> *Sent:* Tuesday, November 03, 2015 18:46
> *To:* Aboud, Amjad
> *Cc:* llvm-dev at lists.llvm.org
> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>
>
>
>
>
>
>
> On Tue, Nov 3, 2015 at 12:16 AM, Aboud, Amjad <amjad.aboud at
intel.com>
> wrote:
>
> > Do we really need to touch the AST? Or would it be reasonable to wire
up
> the CGDebugInfo directly to the PPCallbacks, if it isn't already?
(perhaps
> it is already wired up for other reasons?)
>
> This sound as a good idea, I will check that approach.
>
> PPCallbacks is only an interface, has nothing connected to it, but we will
> create a new class, which implement PPCallbacks, for macros.
>
>
>
> Right - I was wondering if CGDebugInfo already implemented PPCallbacks or
> was otherwise being notified of PPCallback related things, possibly through
> a layer or two of indirection.
>
>
>
> So we can connect whatever we want to that class.
>
> The only drawback with this approach, is that we can test the frontend
> using the generated  LLVM IR, i.e. the whole path, instead of having two
> tests, AST for testing the parser, and LLVM IR for testing the Sema.
>
>
>
> We don't usually do direct AST tests in Clang for debug info (or for
many
> things, really) - we just do source -> llvm IR anyway, so that's
nothing
> out of the ordinary.
>
>
>
>
>
> > I wonder if it'd be better to use a parent chain style approach
(DIMacro
> has a DIMacroFile it refers to, each DIMacroFile has another one that it
> refers to, up to null)?
> > (does it ever make sense/need to have a DIMacroFile without any macros
> in it? I assume not?)
> First, it seems that GCC does emit MacroFile that has no macros inside (I
> understand that it might not be useful, but I am not sure if we should
> ignore that or not).
>
>
>
> Yeah, that's weird - I'd sort of be inclined to skip it until we
know what
> it's useful for.
>
>
>
> Second, I assume that you are suggesting the parent chain style instead to
> the current children style, right?
>
>
>
> Correct
>
>
>
> In this case, won’t it make the debug emitter code much complicated to
> figure out the DFS tree,
>
>
>
> I don't quite imagine it would be more complicated - we would just be
> building the file parent chain as we go, and keeping the current macro file
> around to be used as the parent to any macros we create.
>
>
>
> which should be emitted for the macros, not mentioning the macro order
> which will be lost?
>
>
>
> Not necessarily, if we kept the macros in order in the list of macros
> attached to the CU, which I imagine we would.
>
>
>
> Also, remember that the command line macros have no DIMacroFile parent.
>
>
>
> Fair - they could have the null parent, potentially.
>
>
>
> However, if you meant to use the parent chain in addition to the children
> list, then what extra information it will give us?
>
>
>
> >Might be good to start with dwarfdump support - seems useful regardless
> of anything else?
>
> I agree, and in fact, I already have this code implemented, will upload it
> for review soon.
>
>
>
> Cool
>
>
>
>
>
> Thanks,
>
> Amjad
>
>
>
> *From:* David Blaikie [mailto:dblaikie at gmail.com]
> *Sent:* Tuesday, November 03, 2015 00:32
> *To:* Aboud, Amjad
> *Cc:* llvm-dev at lists.llvm.org
> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>
>
>
>
>
>
>
> On Wed, Oct 28, 2015 at 7:56 AM, Aboud, Amjad via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi,
>
> I would like to implement macro debug info support in LLVM.
>
> Below you will find 4 parts:
>
> 1.      Background  on what does it mean to debug macros.
>
> 2.      A brief explanation on how to represent macro debug info in DWARF
> 4.0.
>
> 3.      The suggested design.
>
> 4.      A full example: Source -> AST -> LLVM IR -> DWARF.
>
>
>
> Feel free to skip first two parts if you think you know the background.
>
> Please, let me know if you have any comment or feedback on this approach.
>
>
>
> Thanks,
>
> Amjad
>
>
>
> *[Background]*
>
> There are two kind of macro definition:
>
> 1. Simple macro definition, e.g.  #define M1 Value1
>
> 2. Function macro definition, e.g. #define M2(x, y)  (x) + (y)
>
> Macro scope starts with the "#define" directive and ends with
"#undef"
> directive.
>
>
>
> GDB supports debugging macros. This means, it can evaluate the macro
> expression for all macros, which have a scope that interleaves with the
> current breakpoint.
>
> For example:
>
> GDB command: print M2(3, 5)
>
> GDB Result: 8
>
>
>
> GDB can evaluate the macro expression based on the
".debug_macroinfo"
> section (DWARF 4.0).
>
>
>
> *[DWARF 4.0 ".debug_macroinfo" section]*
>
> In this section there are 4 kinds of entries
>
> 1.      DW_MACROINFO_define
>
> 2.      DW_MACROINFO_undef
>
> 3.      DW_MACROINFO_start_file
>
> 4.      DW_MACROINFO_end_file
>
>
>
> Note: There is a 5th kind of entry for vendor specific macro information,
> that we do not need to support.
>
>
>
> The first two entries contain information about the line number where the
> macro is defined/undefined, and a null terminated string, which contain the
> macro name (followed by the replacement value in case of a definition, or a
> list of parameters then the replacement value in case of function macro
> definition).
>
> The third entry contains information about the line where the file was
> included followed by the file id (an offset into the files table in the
> debug line section).
>
> The fourth entry contains nothing, and it just close the previous entry of
> third kind (start_file) .
>
>
>
> Macro definition and file including entries must appear at the same order
> as they appear in the source file. Where all macro entries between
> "start_file" and "end_file" entries represent macros
appears
> directly/indirectly in the included file.
>
>
>
> Special cases:
>
> 1.      The main source file should be the first "start_file"
entry in
> the sequence, and should have line number "0".
>
> 2.      Command line/Compiler definitions must also have line number
"0"
> but must appear before the first "start_file" entry.
>
> 3.      Command line include files, must also have line number
"0" but
> will appear straight after the "start_file" of the main source.
>
>
>
> *[Design]*
>
> To support macros the following components need to be modified: Clang,
> LLVM IR, Dwarf Debug emitter.
>
>
>
> In clang, we need to handle these source directives:
>
> 1.      #define
>
> 2.      #undef
>
> 3.      #include
>
> The idea is to make a use of "PPCallbacks" class, which allows
> preprocessor to notify the parser each time one of the above directives
> occurs.
>
> These are the callbacks that should be implemented:
>
> "MacroDefined", "MacroUndefined",
"FileChanged", and "InclusionDirective".
>
>
>
> AST will be extended to support two new DECL types: "MacroDecl"
and
> "FileIncludeDecl".
>
>
>
> Do we really need to touch the AST? Or would it be reasonable to wire up
> the CGDebugInfo directly to the PPCallbacks, if it isn't already?
(perhaps
> it is already wired up for other reasons?)
>
>
>
> Where "FileIncludeDecl" AST might contain other
> "FileIncludeDecl"/"MacroDecl" ASTs.
>
> These two new AST DECLs are not part of TranslationUnitDecl and are
> handled separately (see AST example below).
>
>
>
> In the LLVM IR, metadata debug info will be extended to support new DIs as
> well:
>
> "DIMacro", "DIFileInclude", and "MacroNode".
>
> The last, is needed as we cannot use DINode as a base class of
"DIMacro"
> and DIFileInclude" nodes.
>
>
>
> DIMacro will contain:
>
> ·        type (definition/undefinition).
>
> ·        line number (interger).
>
> ·        name (null terminated string).
>
> ·        replacement value  (null terminated string - optional).
>
>
>
> DIFileMacro will contain:
>
> ·        line number (interger).
>
> ·        file (DIFile).
>
> ·        macro list (MacroNodeArray) - optional.
>
>
>
> I wonder if it'd be better to use a parent chain style approach
(DIMacro
> has a DIMacroFile it refers to, each DIMacroFile has another one that it
> refers to, up to null)?
> (does it ever make sense/need to have a DIMacroFile without any macros in
> it? I assume not?)
>
>
> Might be good to start with dwarfdump support - seems useful regardless of
> anything else?
>
>
>
>
>
> In addition, the DICompileUnit will contain a new optional field of macro
> list of type (MacroNodeArray).
>
>
>
> Finally, I assume that macro support should be disabled by default, and
> there should be a flag to enable this feature. I would say that we should
> introduce a new specific flag, e.g. "-gmacro", that could be used
with
> "-g".
>
>
>
> *[Example]*
>
> Here is an example that demonstrate the macro support from
> Source->AST->LLVM IR->DWARF.
>
>
>
> Source
>
> ========================================================>
> mainfile.c:
>
>
>
--------------------------------------------------------------------------------------
>
> 1. #define M1 Value1
>
> 2. #include "myfile.h"
>
> 3. #define M2( x , y)   ( (x)    + (y)  * Value2)
>
>
>
--------------------------------------------------------------------------------------
>
>
>
> myfile.h:
>
>
>
--------------------------------------------------------------------------------------
>
> 1.
>
> 2.
>
> 3.
>
> 4. #undef M1
>
> 5. #define M1 NewValue1
>
>
>
--------------------------------------------------------------------------------------
>
>
>
> myfile2.h:
>
>
>
--------------------------------------------------------------------------------------
>
> 1. #define M4 Value4
>
>
>
--------------------------------------------------------------------------------------
>
> ========================================================>
>
>
> Command line:
>
> clang -c -g -gmacro -O0 -DM3=Value3 -include myfile2.h mainfile.c
>
>
>
>
>
> AST
>
> ========================================================>
> MacroDecl 0xd6c5c0 <<invalid sloc>> <invalid sloc>
__llvm__ defined
>
> MacroDecl 0xd6c618 <<invalid sloc>> <invalid sloc>
__clang__ defined
>
>
>
> … <More compiler macros> …
>
>
>
> MacroDecl 0x11c01b0 <<invalid sloc>> <invalid sloc> M3
defined
>
> FileIncludeDecl 0x11c0208 <mainfile.c:1:1> col:1
>
> |-FileIncludeDecl 0x11c0238 <myfile2.h:1:1> col:1
>
> | `-MacroDecl 0x11c0268 <<invalid sloc>> <invalid sloc>
M4 defined
>
> |-MacroDecl 0x11c02c0 <mainfile.c:1:9> col:9 M1 defined
>
> |-FileIncludeDecl 0x11c0318 <myfile.h:1:1> col:1
>
> | |-MacroDecl 0x11c0348 <line:4:8> col:8 M1 undefined
>
> | `-MacroDecl 0x11c03a0 <line:5:9> col:9 M1 defined
>
> `-MacroDecl 0x11c03f8 <mainfile.c:3:9> col:9 M2 defined
>
> TranslationUnitDecl 0xd6c078 <<invalid sloc>> <invalid
sloc>
>
> |-TypedefDecl 0xd6c330 <<invalid sloc>> <invalid sloc>
implicit __int128_t
> '__int128'
>
> |-TypedefDecl 0xd6c370 <<invalid sloc>> <invalid sloc>
implicit
> __uint128_t 'unsigned __int128'
>
> |-TypedefDecl 0xd6c3c8 <<invalid sloc>> <invalid sloc>
implicit
> __builtin_ms_va_list 'char *'
>
> `-TypedefDecl 0xd6c590 <<invalid sloc>> <invalid sloc>
implicit
> __builtin_va_list 'struct __va_list_tag [1]'
>
> ========================================================>
>
>
>
>
> LLVM IR
>
> ========================================================>
> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>
> target triple = "x86_64-pc-linux"
>
>
>
> !llvm.dbg.cu = !{!0}
>
> !llvm.module.flags = !{!327}
>
> !llvm.ident = !{!328}
>
>
>
> !0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer:
> "clang version 3.8.0 (trunk 251321)", isOptimized: false,
runtimeVersion:
> 0, emissionKind: 1, enums: !2, macros: !3)
>
> !1 = !DIFile(filename: "mainfile.c", directory: "/")
>
> !2 = !{}
>
> !3 = !{!4, !5, … <More compiler macros> … , !312, !313}
>
> !4 = !DIMacro(macro type: DW_MACINFO_define, name: "__llvm__",
value: !"1")
>
> !5 = !DIMacro(macro type: DW_MACINFO_define, name: "__clang__",
value:
> !"1")
>
>
>
> … <More compiler macros> …
>
>
>
> !312 = !DIMacro(macro type: DW_MACINFO_define, name: "M3", value:
> !"Value3")
>
> !313 = !DIFileInclude(file: !314, nodes: !315)
>
> !314 = !DIFile(filename: "mainfile.c", directory: "/")
>
> !315 = !{!316, !320, !321, !326}
>
> !316 = !DIFileInclude(file: !317, nodes: !318)
>
> !317 = !DIFile(filename: "myfile2.h", directory: "/")
>
> !318 = !{!319}
>
> !319 = !DIMacro(macro type: DW_MACINFO_define, name: "M4", value:
> !"Value4")
>
> !320 = !DIMacro(macro type: DW_MACINFO_define, name: "M1", line:
1, value:
> !"Value1")
>
> !321 = !DIFileInclude(line: 2, file: !322, nodes: !323)
>
> !322 = !DIFile(filename: "myfile.h", directory: "/")
>
> !323 = !{!324, !325}
>
> !324 = !DIMacro(macro type: DW_MACINFO_undef, name: "M1", line:
4)
>
> !325 = !DIMacro(macro type: DW_MACINFO_define, name: "M1", line:
5, value:
> !"NewValue1")
>
> !326 = !DIMacro(macro type: DW_MACINFO_define, name: "M2(x,y)",
line: 3,
> value: !"( (x) + (y) * Value2)")
>
> !327 = !{i32 2, !"Debug Info Version", i32 3}
>
> !328 = !{!"clang version 3.8.0 (trunk 251321)"}
>
> ========================================================>
>
>
>
>
> DWARF
>
> ========================================================>
> Command line: llvm-dwarfdump.exe -debug-dump=macro mainfile.o
>
>
>
--------------------------------------------------------------------------------------
>
> mainfile3.o:  file format ELF64-x86-64
>
>
>
> .debug_macinfo contents:
>
> DW_MACINFO_define - lineno: 0 macro: __llvm__ 1
>
> DW_MACINFO_define - lineno: 0 macro: __clang__ 1
>
>
>
> … <More compiler macros> …
>
>
>
> DW_MACINFO_define - lineno: 0 macro: M3 Value3
>
> DW_MACINFO_start_file - lineno: 0 filenum: 1
>
>   DW_MACINFO_start_file - lineno: 0 filenum: 2
>
>     DW_MACINFO_define - lineno: 0 macro: M4 Value4
>
>   DW_MACINFO_end_file
>
>   DW_MACINFO_define - lineno: 1 macro: M1 Value1
>
>   DW_MACINFO_start_file - lineno: 2 filenum: 3
>
>     DW_MACINFO_undef - lineno: 4 macro: M1
>
>     DW_MACINFO_define - lineno: 5 macro: M1 NewValue1
>
>   DW_MACINFO_end_file
>
>   DW_MACINFO_define - lineno: 3 macro: M2(x,y) ( (x) + (y) * Value2)
>
> DW_MACINFO_end_file
>
>
>
>
>
--------------------------------------------------------------------------------------
>
> Command line: llvm-dwarfdump.exe -debug-dump=line mainfile.o
>
>
>
--------------------------------------------------------------------------------------
>
> .debug_line contents:
>
>
>
> … <Other line table Info> …
>
>
>
>                 Dir  Mod Time   File Len   File Name
>
>                 ---- ---------- ---------- ---------------------------
>
> file_names[  1]    1 0x00000000 0x00000000 mainfile.c
>
> file_names[  2]    1 0x00000000 0x00000000 myfile2.h
>
> file_names[  3]    1 0x00000000 0x00000000 myfile.h
>
> ========================================================>
>
>
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>
>
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151113/e63b0a0e/attachment-0001.html>

On Fri, Nov 13, 2015 at 10:21 AM, David Blaikie via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> On Mon, Nov 9, 2015 at 4:00 AM, Aboud, Amjad <amjad.aboud at
intel.com>
> wrote:
>
>> I found a way to skip representing macros in AST and create them
directly
>> in CGDebugInfo through PPCallbacks during preprocessing.
>>
>> To do that, I needed to extend ASTConsumer interface with this extra
>> method:
>>
>>
>>
>>   /// If the consumer is interested in notifications from Preprocessor,
>>
>>   /// for example: notifications on macro definitions, etc., it should
>> return
>>
>>   /// a pointer to a PPCallbacks here.
>>
>>   /// The caller takes ownership on the returned pointer.
>>
>>   virtual PPCallbacks *CreatePreprocessorCallbacks() { return nullptr;
}
>>
>>
>>
>> Then the ParseAST can use it to add these preprocessor callbacks, which
>> are needed by the AST consumer, to the preprocessor:
>>
>>
>>
>>   S.getPreprocessor().addPPCallbacks(
>>
>>       std::unique_ptr<PPCallbacks
>> >(Consumer->CreatePreprocessorCallbacks()));
>>
>
> (CreatePreprocessorCallbacks, if that's the path we take, should return
a
> unique_ptr itself rather than returning a raw ownership-passing pointer,
> but that's a minor API detail)
>
>
>>
>>
>> With this, approach the change in clang to support macros is very
small.
>>
>>
>>
>> Do you agree to this approach?
>>
>
> Richard - what do you reckon's the right hook/path to get preprocessor
> info through to codegen (& CGDebugInfo in particular). Would a general
> purpose hook in the ASTConsumer be appropriate/useful?
>
ASTConsumer shouldn't know anything about the preprocessor; there's no
reason to think, in general, that the AST is being produced by
preprocessing and parsing some text. Perhaps adding a PreprocessorConsumer
interface akin to the existing SemaConsumer interface would be a better way
to go.

Thanks,
>>
>> Amjad
>>
>>
>>
>> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf
Of *Aboud,
>> Amjad via llvm-dev
>> *Sent:* Thursday, November 05, 2015 16:56
>> *To:* David Blaikie
>>
>> *Cc:* llvm-dev at lists.llvm.org
>> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>>
>>
>>
>> > Right - I was wondering if CGDebugInfo already implemented
PPCallbacks
>> or was otherwise being notified of PPCallback related things, possibly
>> through a layer or two of indirection.
>>
>>
>>
>> I checked the approach of skipping representing macros in AST, and
>> communicate them directly from Parser to CGDebugInfo.
>>
>> However, I could not find a way to initialize this communication.
>>
>> The only interface available through Parser is either Sema (to create
an
>> AST) or ASTConsumer. While the CGDebugInfo is only available in the
>> CodeGenModule, which is accessible from BackendConsumer, that
implements
>> ASTConsumer.
>>
>>
>>
>> David, skipping the AST will save a lot of code, but I need help
figuring
>> out how to communicate with the CGDebugInfo.
>>
>>
>>
>> Thanks,
>>
>> Amjad
>>
>>
>>
>> *From:* David Blaikie [mailto:dblaikie at gmail.com <dblaikie at
gmail.com>]
>> *Sent:* Tuesday, November 03, 2015 18:46
>> *To:* Aboud, Amjad
>> *Cc:* llvm-dev at lists.llvm.org
>> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Nov 3, 2015 at 12:16 AM, Aboud, Amjad <amjad.aboud at
intel.com>
>> wrote:
>>
>> > Do we really need to touch the AST? Or would it be reasonable to
wire
>> up the CGDebugInfo directly to the PPCallbacks, if it isn't
already?
>> (perhaps it is already wired up for other reasons?)
>>
>> This sound as a good idea, I will check that approach.
>>
>> PPCallbacks is only an interface, has nothing connected to it, but we
>> will create a new class, which implement PPCallbacks, for macros.
>>
>>
>>
>> Right - I was wondering if CGDebugInfo already implemented PPCallbacks
or
>> was otherwise being notified of PPCallback related things, possibly
through
>> a layer or two of indirection.
>>
>>
>>
>> So we can connect whatever we want to that class.
>>
>> The only drawback with this approach, is that we can test the frontend
>> using the generated  LLVM IR, i.e. the whole path, instead of having
two
>> tests, AST for testing the parser, and LLVM IR for testing the Sema.
>>
>>
>>
>> We don't usually do direct AST tests in Clang for debug info (or
for many
>> things, really) - we just do source -> llvm IR anyway, so that's
nothing
>> out of the ordinary.
>>
>>
>>
>>
>>
>> > I wonder if it'd be better to use a parent chain style
approach
>> (DIMacro has a DIMacroFile it refers to, each DIMacroFile has another
one
>> that it refers to, up to null)?
>> > (does it ever make sense/need to have a DIMacroFile without any
macros
>> in it? I assume not?)
>> First, it seems that GCC does emit MacroFile that has no macros inside
(I
>> understand that it might not be useful, but I am not sure if we should
>> ignore that or not).
>>
>>
>>
>> Yeah, that's weird - I'd sort of be inclined to skip it until
we know
>> what it's useful for.
>>
>>
>>
>> Second, I assume that you are suggesting the parent chain style instead
>> to the current children style, right?
>>
>>
>>
>> Correct
>>
>>
>>
>> In this case, won’t it make the debug emitter code much complicated to
>> figure out the DFS tree,
>>
>>
>>
>> I don't quite imagine it would be more complicated - we would just
be
>> building the file parent chain as we go, and keeping the current macro
file
>> around to be used as the parent to any macros we create.
>>
>>
>>
>> which should be emitted for the macros, not mentioning the macro order
>> which will be lost?
>>
>>
>>
>> Not necessarily, if we kept the macros in order in the list of macros
>> attached to the CU, which I imagine we would.
>>
>>
>>
>> Also, remember that the command line macros have no DIMacroFile parent.
>>
>>
>>
>> Fair - they could have the null parent, potentially.
>>
>>
>>
>> However, if you meant to use the parent chain in addition to the
children
>> list, then what extra information it will give us?
>>
>>
>>
>> >Might be good to start with dwarfdump support - seems useful
regardless
>> of anything else?
>>
>> I agree, and in fact, I already have this code implemented, will upload
>> it for review soon.
>>
>>
>>
>> Cool
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Amjad
>>
>>
>>
>> *From:* David Blaikie [mailto:dblaikie at gmail.com]
>> *Sent:* Tuesday, November 03, 2015 00:32
>> *To:* Aboud, Amjad
>> *Cc:* llvm-dev at lists.llvm.org
>> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Oct 28, 2015 at 7:56 AM, Aboud, Amjad via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> Hi,
>>
>> I would like to implement macro debug info support in LLVM.
>>
>> Below you will find 4 parts:
>>
>> 1.      Background  on what does it mean to debug macros.
>>
>> 2.      A brief explanation on how to represent macro debug info in
>> DWARF 4.0.
>>
>> 3.      The suggested design.
>>
>> 4.      A full example: Source -> AST -> LLVM IR -> DWARF.
>>
>>
>>
>> Feel free to skip first two parts if you think you know the background.
>>
>> Please, let me know if you have any comment or feedback on this
approach.
>>
>>
>>
>> Thanks,
>>
>> Amjad
>>
>>
>>
>> *[Background]*
>>
>> There are two kind of macro definition:
>>
>> 1. Simple macro definition, e.g.  #define M1 Value1
>>
>> 2. Function macro definition, e.g. #define M2(x, y)  (x) + (y)
>>
>> Macro scope starts with the "#define" directive and ends with
"#undef"
>> directive.
>>
>>
>>
>> GDB supports debugging macros. This means, it can evaluate the macro
>> expression for all macros, which have a scope that interleaves with the
>> current breakpoint.
>>
>> For example:
>>
>> GDB command: print M2(3, 5)
>>
>> GDB Result: 8
>>
>>
>>
>> GDB can evaluate the macro expression based on the
".debug_macroinfo"
>> section (DWARF 4.0).
>>
>>
>>
>> *[DWARF 4.0 ".debug_macroinfo" section]*
>>
>> In this section there are 4 kinds of entries
>>
>> 1.      DW_MACROINFO_define
>>
>> 2.      DW_MACROINFO_undef
>>
>> 3.      DW_MACROINFO_start_file
>>
>> 4.      DW_MACROINFO_end_file
>>
>>
>>
>> Note: There is a 5th kind of entry for vendor specific macro
information,
>> that we do not need to support.
>>
>>
>>
>> The first two entries contain information about the line number where
the
>> macro is defined/undefined, and a null terminated string, which contain
the
>> macro name (followed by the replacement value in case of a definition,
or a
>> list of parameters then the replacement value in case of function macro
>> definition).
>>
>> The third entry contains information about the line where the file was
>> included followed by the file id (an offset into the files table in the
>> debug line section).
>>
>> The fourth entry contains nothing, and it just close the previous entry
>> of third kind (start_file) .
>>
>>
>>
>> Macro definition and file including entries must appear at the same
order
>> as they appear in the source file. Where all macro entries between
>> "start_file" and "end_file" entries represent
macros appears
>> directly/indirectly in the included file.
>>
>>
>>
>> Special cases:
>>
>> 1.      The main source file should be the first "start_file"
entry in
>> the sequence, and should have line number "0".
>>
>> 2.      Command line/Compiler definitions must also have line number
"0"
>> but must appear before the first "start_file" entry.
>>
>> 3.      Command line include files, must also have line number
"0" but
>> will appear straight after the "start_file" of the main
source.
>>
>>
>>
>> *[Design]*
>>
>> To support macros the following components need to be modified: Clang,
>> LLVM IR, Dwarf Debug emitter.
>>
>>
>>
>> In clang, we need to handle these source directives:
>>
>> 1.      #define
>>
>> 2.      #undef
>>
>> 3.      #include
>>
>> The idea is to make a use of "PPCallbacks" class, which
allows
>> preprocessor to notify the parser each time one of the above directives
>> occurs.
>>
>> These are the callbacks that should be implemented:
>>
>> "MacroDefined", "MacroUndefined",
"FileChanged", and "InclusionDirective".
>>
>>
>>
>> AST will be extended to support two new DECL types:
"MacroDecl" and
>> "FileIncludeDecl".
>>
>>
>>
>> Do we really need to touch the AST? Or would it be reasonable to wire
up
>> the CGDebugInfo directly to the PPCallbacks, if it isn't already?
(perhaps
>> it is already wired up for other reasons?)
>>
>>
>>
>> Where "FileIncludeDecl" AST might contain other
>> "FileIncludeDecl"/"MacroDecl" ASTs.
>>
>> These two new AST DECLs are not part of TranslationUnitDecl and are
>> handled separately (see AST example below).
>>
>>
>>
>> In the LLVM IR, metadata debug info will be extended to support new DIs
>> as well:
>>
>> "DIMacro", "DIFileInclude", and
"MacroNode".
>>
>> The last, is needed as we cannot use DINode as a base class of
"DIMacro"
>> and DIFileInclude" nodes.
>>
>>
>>
>> DIMacro will contain:
>>
>> ·        type (definition/undefinition).
>>
>> ·        line number (interger).
>>
>> ·        name (null terminated string).
>>
>> ·        replacement value  (null terminated string - optional).
>>
>>
>>
>> DIFileMacro will contain:
>>
>> ·        line number (interger).
>>
>> ·        file (DIFile).
>>
>> ·        macro list (MacroNodeArray) - optional.
>>
>>
>>
>> I wonder if it'd be better to use a parent chain style approach
(DIMacro
>> has a DIMacroFile it refers to, each DIMacroFile has another one that
it
>> refers to, up to null)?
>> (does it ever make sense/need to have a DIMacroFile without any macros
in
>> it? I assume not?)
>>
>>
>> Might be good to start with dwarfdump support - seems useful regardless
>> of anything else?
>>
>>
>>
>>
>>
>> In addition, the DICompileUnit will contain a new optional field of
macro
>> list of type (MacroNodeArray).
>>
>>
>>
>> Finally, I assume that macro support should be disabled by default, and
>> there should be a flag to enable this feature. I would say that we
should
>> introduce a new specific flag, e.g. "-gmacro", that could be
used with
>> "-g".
>>
>>
>>
>> *[Example]*
>>
>> Here is an example that demonstrate the macro support from
>> Source->AST->LLVM IR->DWARF.
>>
>>
>>
>> Source
>>
>> ========================================================>>
>> mainfile.c:
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>> 1. #define M1 Value1
>>
>> 2. #include "myfile.h"
>>
>> 3. #define M2( x , y)   ( (x)    + (y)  * Value2)
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>>
>>
>> myfile.h:
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>> 1.
>>
>> 2.
>>
>> 3.
>>
>> 4. #undef M1
>>
>> 5. #define M1 NewValue1
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>>
>>
>> myfile2.h:
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>> 1. #define M4 Value4
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>> ========================================================>>
>>
>>
>> Command line:
>>
>> clang -c -g -gmacro -O0 -DM3=Value3 -include myfile2.h mainfile.c
>>
>>
>>
>>
>>
>> AST
>>
>> ========================================================>>
>> MacroDecl 0xd6c5c0 <<invalid sloc>> <invalid sloc>
__llvm__ defined
>>
>> MacroDecl 0xd6c618 <<invalid sloc>> <invalid sloc>
__clang__ defined
>>
>>
>>
>> … <More compiler macros> …
>>
>>
>>
>> MacroDecl 0x11c01b0 <<invalid sloc>> <invalid sloc>
M3 defined
>>
>> FileIncludeDecl 0x11c0208 <mainfile.c:1:1> col:1
>>
>> |-FileIncludeDecl 0x11c0238 <myfile2.h:1:1> col:1
>>
>> | `-MacroDecl 0x11c0268 <<invalid sloc>> <invalid
sloc> M4 defined
>>
>> |-MacroDecl 0x11c02c0 <mainfile.c:1:9> col:9 M1 defined
>>
>> |-FileIncludeDecl 0x11c0318 <myfile.h:1:1> col:1
>>
>> | |-MacroDecl 0x11c0348 <line:4:8> col:8 M1 undefined
>>
>> | `-MacroDecl 0x11c03a0 <line:5:9> col:9 M1 defined
>>
>> `-MacroDecl 0x11c03f8 <mainfile.c:3:9> col:9 M2 defined
>>
>> TranslationUnitDecl 0xd6c078 <<invalid sloc>> <invalid
sloc>
>>
>> |-TypedefDecl 0xd6c330 <<invalid sloc>> <invalid
sloc> implicit
>> __int128_t '__int128'
>>
>> |-TypedefDecl 0xd6c370 <<invalid sloc>> <invalid
sloc> implicit
>> __uint128_t 'unsigned __int128'
>>
>> |-TypedefDecl 0xd6c3c8 <<invalid sloc>> <invalid
sloc> implicit
>> __builtin_ms_va_list 'char *'
>>
>> `-TypedefDecl 0xd6c590 <<invalid sloc>> <invalid
sloc> implicit
>> __builtin_va_list 'struct __va_list_tag [1]'
>>
>> ========================================================>>
>>
>>
>>
>>
>> LLVM IR
>>
>> ========================================================>>
>> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>
>> target triple = "x86_64-pc-linux"
>>
>>
>>
>> !llvm.dbg.cu = !{!0}
>>
>> !llvm.module.flags = !{!327}
>>
>> !llvm.ident = !{!328}
>>
>>
>>
>> !0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer:
>> "clang version 3.8.0 (trunk 251321)", isOptimized: false,
runtimeVersion:
>> 0, emissionKind: 1, enums: !2, macros: !3)
>>
>> !1 = !DIFile(filename: "mainfile.c", directory:
"/")
>>
>> !2 = !{}
>>
>> !3 = !{!4, !5, … <More compiler macros> … , !312, !313}
>>
>> !4 = !DIMacro(macro type: DW_MACINFO_define, name:
"__llvm__", value:
>> !"1")
>>
>> !5 = !DIMacro(macro type: DW_MACINFO_define, name:
"__clang__", value:
>> !"1")
>>
>>
>>
>> … <More compiler macros> …
>>
>>
>>
>> !312 = !DIMacro(macro type: DW_MACINFO_define, name: "M3",
value:
>> !"Value3")
>>
>> !313 = !DIFileInclude(file: !314, nodes: !315)
>>
>> !314 = !DIFile(filename: "mainfile.c", directory:
"/")
>>
>> !315 = !{!316, !320, !321, !326}
>>
>> !316 = !DIFileInclude(file: !317, nodes: !318)
>>
>> !317 = !DIFile(filename: "myfile2.h", directory:
"/")
>>
>> !318 = !{!319}
>>
>> !319 = !DIMacro(macro type: DW_MACINFO_define, name: "M4",
value:
>> !"Value4")
>>
>> !320 = !DIMacro(macro type: DW_MACINFO_define, name: "M1",
line: 1,
>> value: !"Value1")
>>
>> !321 = !DIFileInclude(line: 2, file: !322, nodes: !323)
>>
>> !322 = !DIFile(filename: "myfile.h", directory:
"/")
>>
>> !323 = !{!324, !325}
>>
>> !324 = !DIMacro(macro type: DW_MACINFO_undef, name: "M1",
line: 4)
>>
>> !325 = !DIMacro(macro type: DW_MACINFO_define, name: "M1",
line: 5,
>> value: !"NewValue1")
>>
>> !326 = !DIMacro(macro type: DW_MACINFO_define, name:
"M2(x,y)", line: 3,
>> value: !"( (x) + (y) * Value2)")
>>
>> !327 = !{i32 2, !"Debug Info Version", i32 3}
>>
>> !328 = !{!"clang version 3.8.0 (trunk 251321)"}
>>
>> ========================================================>>
>>
>>
>>
>>
>> DWARF
>>
>> ========================================================>>
>> Command line: llvm-dwarfdump.exe -debug-dump=macro mainfile.o
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>> mainfile3.o:  file format ELF64-x86-64
>>
>>
>>
>> .debug_macinfo contents:
>>
>> DW_MACINFO_define - lineno: 0 macro: __llvm__ 1
>>
>> DW_MACINFO_define - lineno: 0 macro: __clang__ 1
>>
>>
>>
>> … <More compiler macros> …
>>
>>
>>
>> DW_MACINFO_define - lineno: 0 macro: M3 Value3
>>
>> DW_MACINFO_start_file - lineno: 0 filenum: 1
>>
>>   DW_MACINFO_start_file - lineno: 0 filenum: 2
>>
>>     DW_MACINFO_define - lineno: 0 macro: M4 Value4
>>
>>   DW_MACINFO_end_file
>>
>>   DW_MACINFO_define - lineno: 1 macro: M1 Value1
>>
>>   DW_MACINFO_start_file - lineno: 2 filenum: 3
>>
>>     DW_MACINFO_undef - lineno: 4 macro: M1
>>
>>     DW_MACINFO_define - lineno: 5 macro: M1 NewValue1
>>
>>   DW_MACINFO_end_file
>>
>>   DW_MACINFO_define - lineno: 3 macro: M2(x,y) ( (x) + (y) * Value2)
>>
>> DW_MACINFO_end_file
>>
>>
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>> Command line: llvm-dwarfdump.exe -debug-dump=line mainfile.o
>>
>>
>>
--------------------------------------------------------------------------------------
>>
>> .debug_line contents:
>>
>>
>>
>> … <Other line table Info> …
>>
>>
>>
>>                 Dir  Mod Time   File Len   File Name
>>
>>                 ---- ---------- ---------- ---------------------------
>>
>> file_names[  1]    1 0x00000000 0x00000000 mainfile.c
>>
>> file_names[  2]    1 0x00000000 0x00000000 myfile2.h
>>
>> file_names[  3]    1 0x00000000 0x00000000 myfile.h
>>
>> ========================================================>>
>>
>>
>> ---------------------------------------------------------------------
>> Intel Israel (74) Limited
>>
>> This e-mail and any attachments may contain confidential material for
>> the sole use of the intended recipient(s). Any review or distribution
>> by others is strictly prohibited. If you are not the intended
>> recipient, please contact the sender and delete all copies.
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>>
>> ---------------------------------------------------------------------
>> Intel Israel (74) Limited
>>
>> This e-mail and any attachments may contain confidential material for
>> the sole use of the intended recipient(s). Any review or distribution
>> by others is strictly prohibited. If you are not the intended
>> recipient, please contact the sender and delete all copies.
>>
>>
>>
>> ---------------------------------------------------------------------
>> Intel Israel (74) Limited
>>
>> This e-mail and any attachments may contain confidential material for
>> the sole use of the intended recipient(s). Any review or distribution
>> by others is strictly prohibited. If you are not the intended
>> recipient, please contact the sender and delete all copies.
>>
>> ---------------------------------------------------------------------
>> Intel Israel (74) Limited
>>
>> This e-mail and any attachments may contain confidential material for
>> the sole use of the intended recipient(s). Any review or distribution
>> by others is strictly prohibited. If you are not the intended
>> recipient, please contact the sender and delete all copies.
>>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151113/b2f80294/attachment.html>

On Fri, Nov 13, 2015 at 2:41 PM, Richard Smith <richard at metafoo.co.uk>
wrote:
> On Fri, Nov 13, 2015 at 10:21 AM, David Blaikie via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> On Mon, Nov 9, 2015 at 4:00 AM, Aboud, Amjad <amjad.aboud at
intel.com>
>> wrote:
>>
>>> I found a way to skip representing macros in AST and create them
>>> directly in CGDebugInfo through PPCallbacks during preprocessing.
>>>
>>> To do that, I needed to extend ASTConsumer interface with this
extra
>>> method:
>>>
>>>
>>>
>>>   /// If the consumer is interested in notifications from
Preprocessor,
>>>
>>>   /// for example: notifications on macro definitions, etc., it
should
>>> return
>>>
>>>   /// a pointer to a PPCallbacks here.
>>>
>>>   /// The caller takes ownership on the returned pointer.
>>>
>>>   virtual PPCallbacks *CreatePreprocessorCallbacks() { return
nullptr; }
>>>
>>>
>>>
>>> Then the ParseAST can use it to add these preprocessor callbacks,
which
>>> are needed by the AST consumer, to the preprocessor:
>>>
>>>
>>>
>>>   S.getPreprocessor().addPPCallbacks(
>>>
>>>       std::unique_ptr<PPCallbacks
>>> >(Consumer->CreatePreprocessorCallbacks()));
>>>
>>
>> (CreatePreprocessorCallbacks, if that's the path we take, should
return a
>> unique_ptr itself rather than returning a raw ownership-passing
pointer,
>> but that's a minor API detail)
>>
>>
>>>
>>>
>>> With this, approach the change in clang to support macros is very
small.
>>>
>>>
>>>
>>> Do you agree to this approach?
>>>
>>
>> Richard - what do you reckon's the right hook/path to get
preprocessor
>> info through to codegen (& CGDebugInfo in particular). Would a
general
>> purpose hook in the ASTConsumer be appropriate/useful?
>>
>
> ASTConsumer shouldn't know anything about the preprocessor; there's
no
> reason to think, in general, that the AST is being produced by
> preprocessing and parsing some text.
>
Hmm, I suppose a fair question then - would it be possible to implement
debug info for macros when reading ASTs from a serialized file (without a
preprocessor). Or should we actually go with the original proposal of
creating AST nodes for the preprocessor events so we can have access to
them after loading serialized modules & then generating debug info from
them? Is there some other side table we'd be better off using/creating for
this task?

> Perhaps adding a PreprocessorConsumer interface akin to the existing
> SemaConsumer interface would be a better way to go.
>
> Thanks,
>>>
>>> Amjad
>>>
>>>
>>>
>>> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On
Behalf Of
>>> *Aboud, Amjad via llvm-dev
>>> *Sent:* Thursday, November 05, 2015 16:56
>>> *To:* David Blaikie
>>>
>>> *Cc:* llvm-dev at lists.llvm.org
>>> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>>>
>>>
>>>
>>> > Right - I was wondering if CGDebugInfo already implemented
>>> PPCallbacks or was otherwise being notified of PPCallback related
things,
>>> possibly through a layer or two of indirection.
>>>
>>>
>>>
>>> I checked the approach of skipping representing macros in AST, and
>>> communicate them directly from Parser to CGDebugInfo.
>>>
>>> However, I could not find a way to initialize this communication.
>>>
>>> The only interface available through Parser is either Sema (to
create an
>>> AST) or ASTConsumer. While the CGDebugInfo is only available in the
>>> CodeGenModule, which is accessible from BackendConsumer, that
implements
>>> ASTConsumer.
>>>
>>>
>>>
>>> David, skipping the AST will save a lot of code, but I need help
>>> figuring out how to communicate with the CGDebugInfo.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Amjad
>>>
>>>
>>>
>>> *From:* David Blaikie [mailto:dblaikie at gmail.com <dblaikie at
gmail.com>]
>>> *Sent:* Tuesday, November 03, 2015 18:46
>>> *To:* Aboud, Amjad
>>> *Cc:* llvm-dev at lists.llvm.org
>>> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Nov 3, 2015 at 12:16 AM, Aboud, Amjad <amjad.aboud at
intel.com>
>>> wrote:
>>>
>>> > Do we really need to touch the AST? Or would it be reasonable
to wire
>>> up the CGDebugInfo directly to the PPCallbacks, if it isn't
already?
>>> (perhaps it is already wired up for other reasons?)
>>>
>>> This sound as a good idea, I will check that approach.
>>>
>>> PPCallbacks is only an interface, has nothing connected to it, but
we
>>> will create a new class, which implement PPCallbacks, for macros.
>>>
>>>
>>>
>>> Right - I was wondering if CGDebugInfo already implemented
PPCallbacks
>>> or was otherwise being notified of PPCallback related things,
possibly
>>> through a layer or two of indirection.
>>>
>>>
>>>
>>> So we can connect whatever we want to that class.
>>>
>>> The only drawback with this approach, is that we can test the
frontend
>>> using the generated  LLVM IR, i.e. the whole path, instead of
having two
>>> tests, AST for testing the parser, and LLVM IR for testing the
Sema.
>>>
>>>
>>>
>>> We don't usually do direct AST tests in Clang for debug info
(or for
>>> many things, really) - we just do source -> llvm IR anyway, so
that's
>>> nothing out of the ordinary.
>>>
>>>
>>>
>>>
>>>
>>> > I wonder if it'd be better to use a parent chain style
approach
>>> (DIMacro has a DIMacroFile it refers to, each DIMacroFile has
another one
>>> that it refers to, up to null)?
>>> > (does it ever make sense/need to have a DIMacroFile without
any macros
>>> in it? I assume not?)
>>> First, it seems that GCC does emit MacroFile that has no macros
inside
>>> (I understand that it might not be useful, but I am not sure if we
should
>>> ignore that or not).
>>>
>>>
>>>
>>> Yeah, that's weird - I'd sort of be inclined to skip it
until we know
>>> what it's useful for.
>>>
>>>
>>>
>>> Second, I assume that you are suggesting the parent chain style
instead
>>> to the current children style, right?
>>>
>>>
>>>
>>> Correct
>>>
>>>
>>>
>>> In this case, won’t it make the debug emitter code much complicated
to
>>> figure out the DFS tree,
>>>
>>>
>>>
>>> I don't quite imagine it would be more complicated - we would
just be
>>> building the file parent chain as we go, and keeping the current
macro file
>>> around to be used as the parent to any macros we create.
>>>
>>>
>>>
>>> which should be emitted for the macros, not mentioning the macro
order
>>> which will be lost?
>>>
>>>
>>>
>>> Not necessarily, if we kept the macros in order in the list of
macros
>>> attached to the CU, which I imagine we would.
>>>
>>>
>>>
>>> Also, remember that the command line macros have no DIMacroFile
parent.
>>>
>>>
>>>
>>> Fair - they could have the null parent, potentially.
>>>
>>>
>>>
>>> However, if you meant to use the parent chain in addition to the
>>> children list, then what extra information it will give us?
>>>
>>>
>>>
>>> >Might be good to start with dwarfdump support - seems useful
regardless
>>> of anything else?
>>>
>>> I agree, and in fact, I already have this code implemented, will
upload
>>> it for review soon.
>>>
>>>
>>>
>>> Cool
>>>
>>>
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Amjad
>>>
>>>
>>>
>>> *From:* David Blaikie [mailto:dblaikie at gmail.com]
>>> *Sent:* Tuesday, November 03, 2015 00:32
>>> *To:* Aboud, Amjad
>>> *Cc:* llvm-dev at lists.llvm.org
>>> *Subject:* Re: [llvm-dev] RFC: Supporting macros in LLVM debug info
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Oct 28, 2015 at 7:56 AM, Aboud, Amjad via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>> Hi,
>>>
>>> I would like to implement macro debug info support in LLVM.
>>>
>>> Below you will find 4 parts:
>>>
>>> 1.      Background  on what does it mean to debug macros.
>>>
>>> 2.      A brief explanation on how to represent macro debug info in
>>> DWARF 4.0.
>>>
>>> 3.      The suggested design.
>>>
>>> 4.      A full example: Source -> AST -> LLVM IR -> DWARF.
>>>
>>>
>>>
>>> Feel free to skip first two parts if you think you know the
background.
>>>
>>> Please, let me know if you have any comment or feedback on this
approach.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Amjad
>>>
>>>
>>>
>>> *[Background]*
>>>
>>> There are two kind of macro definition:
>>>
>>> 1. Simple macro definition, e.g.  #define M1 Value1
>>>
>>> 2. Function macro definition, e.g. #define M2(x, y)  (x) + (y)
>>>
>>> Macro scope starts with the "#define" directive and ends
with "#undef"
>>> directive.
>>>
>>>
>>>
>>> GDB supports debugging macros. This means, it can evaluate the
macro
>>> expression for all macros, which have a scope that interleaves with
the
>>> current breakpoint.
>>>
>>> For example:
>>>
>>> GDB command: print M2(3, 5)
>>>
>>> GDB Result: 8
>>>
>>>
>>>
>>> GDB can evaluate the macro expression based on the
".debug_macroinfo"
>>> section (DWARF 4.0).
>>>
>>>
>>>
>>> *[DWARF 4.0 ".debug_macroinfo" section]*
>>>
>>> In this section there are 4 kinds of entries
>>>
>>> 1.      DW_MACROINFO_define
>>>
>>> 2.      DW_MACROINFO_undef
>>>
>>> 3.      DW_MACROINFO_start_file
>>>
>>> 4.      DW_MACROINFO_end_file
>>>
>>>
>>>
>>> Note: There is a 5th kind of entry for vendor specific macro
>>> information, that we do not need to support.
>>>
>>>
>>>
>>> The first two entries contain information about the line number
where
>>> the macro is defined/undefined, and a null terminated string, which
contain
>>> the macro name (followed by the replacement value in case of a
definition,
>>> or a list of parameters then the replacement value in case of
function
>>> macro definition).
>>>
>>> The third entry contains information about the line where the file
was
>>> included followed by the file id (an offset into the files table in
the
>>> debug line section).
>>>
>>> The fourth entry contains nothing, and it just close the previous
entry
>>> of third kind (start_file) .
>>>
>>>
>>>
>>> Macro definition and file including entries must appear at the same
>>> order as they appear in the source file. Where all macro entries
between
>>> "start_file" and "end_file" entries represent
macros appears
>>> directly/indirectly in the included file.
>>>
>>>
>>>
>>> Special cases:
>>>
>>> 1.      The main source file should be the first
"start_file" entry in
>>> the sequence, and should have line number "0".
>>>
>>> 2.      Command line/Compiler definitions must also have line
number
>>> "0" but must appear before the first
"start_file" entry.
>>>
>>> 3.      Command line include files, must also have line number
"0" but
>>> will appear straight after the "start_file" of the main
source.
>>>
>>>
>>>
>>> *[Design]*
>>>
>>> To support macros the following components need to be modified:
Clang,
>>> LLVM IR, Dwarf Debug emitter.
>>>
>>>
>>>
>>> In clang, we need to handle these source directives:
>>>
>>> 1.      #define
>>>
>>> 2.      #undef
>>>
>>> 3.      #include
>>>
>>> The idea is to make a use of "PPCallbacks" class, which
allows
>>> preprocessor to notify the parser each time one of the above
directives
>>> occurs.
>>>
>>> These are the callbacks that should be implemented:
>>>
>>> "MacroDefined", "MacroUndefined",
"FileChanged", and
>>> "InclusionDirective".
>>>
>>>
>>>
>>> AST will be extended to support two new DECL types:
"MacroDecl" and
>>> "FileIncludeDecl".
>>>
>>>
>>>
>>> Do we really need to touch the AST? Or would it be reasonable to
wire up
>>> the CGDebugInfo directly to the PPCallbacks, if it isn't
already? (perhaps
>>> it is already wired up for other reasons?)
>>>
>>>
>>>
>>> Where "FileIncludeDecl" AST might contain other
>>> "FileIncludeDecl"/"MacroDecl" ASTs.
>>>
>>> These two new AST DECLs are not part of TranslationUnitDecl and are
>>> handled separately (see AST example below).
>>>
>>>
>>>
>>> In the LLVM IR, metadata debug info will be extended to support new
DIs
>>> as well:
>>>
>>> "DIMacro", "DIFileInclude", and
"MacroNode".
>>>
>>> The last, is needed as we cannot use DINode as a base class of
"DIMacro"
>>> and DIFileInclude" nodes.
>>>
>>>
>>>
>>> DIMacro will contain:
>>>
>>> ·        type (definition/undefinition).
>>>
>>> ·        line number (interger).
>>>
>>> ·        name (null terminated string).
>>>
>>> ·        replacement value  (null terminated string - optional).
>>>
>>>
>>>
>>> DIFileMacro will contain:
>>>
>>> ·        line number (interger).
>>>
>>> ·        file (DIFile).
>>>
>>> ·        macro list (MacroNodeArray) - optional.
>>>
>>>
>>>
>>> I wonder if it'd be better to use a parent chain style approach
(DIMacro
>>> has a DIMacroFile it refers to, each DIMacroFile has another one
that it
>>> refers to, up to null)?
>>> (does it ever make sense/need to have a DIMacroFile without any
macros
>>> in it? I assume not?)
>>>
>>>
>>> Might be good to start with dwarfdump support - seems useful
regardless
>>> of anything else?
>>>
>>>
>>>
>>>
>>>
>>> In addition, the DICompileUnit will contain a new optional field of
>>> macro list of type (MacroNodeArray).
>>>
>>>
>>>
>>> Finally, I assume that macro support should be disabled by default,
and
>>> there should be a flag to enable this feature. I would say that we
should
>>> introduce a new specific flag, e.g. "-gmacro", that could
be used with
>>> "-g".
>>>
>>>
>>>
>>> *[Example]*
>>>
>>> Here is an example that demonstrate the macro support from
>>> Source->AST->LLVM IR->DWARF.
>>>
>>>
>>>
>>> Source
>>>
>>>
========================================================>>>
>>> mainfile.c:
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>> 1. #define M1 Value1
>>>
>>> 2. #include "myfile.h"
>>>
>>> 3. #define M2( x , y)   ( (x)    + (y)  * Value2)
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>>
>>>
>>> myfile.h:
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>> 1.
>>>
>>> 2.
>>>
>>> 3.
>>>
>>> 4. #undef M1
>>>
>>> 5. #define M1 NewValue1
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>>
>>>
>>> myfile2.h:
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>> 1. #define M4 Value4
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>>
========================================================>>>
>>>
>>>
>>> Command line:
>>>
>>> clang -c -g -gmacro -O0 -DM3=Value3 -include myfile2.h mainfile.c
>>>
>>>
>>>
>>>
>>>
>>> AST
>>>
>>>
========================================================>>>
>>> MacroDecl 0xd6c5c0 <<invalid sloc>> <invalid
sloc> __llvm__ defined
>>>
>>> MacroDecl 0xd6c618 <<invalid sloc>> <invalid
sloc> __clang__ defined
>>>
>>>
>>>
>>> … <More compiler macros> …
>>>
>>>
>>>
>>> MacroDecl 0x11c01b0 <<invalid sloc>> <invalid
sloc> M3 defined
>>>
>>> FileIncludeDecl 0x11c0208 <mainfile.c:1:1> col:1
>>>
>>> |-FileIncludeDecl 0x11c0238 <myfile2.h:1:1> col:1
>>>
>>> | `-MacroDecl 0x11c0268 <<invalid sloc>> <invalid
sloc> M4 defined
>>>
>>> |-MacroDecl 0x11c02c0 <mainfile.c:1:9> col:9 M1 defined
>>>
>>> |-FileIncludeDecl 0x11c0318 <myfile.h:1:1> col:1
>>>
>>> | |-MacroDecl 0x11c0348 <line:4:8> col:8 M1 undefined
>>>
>>> | `-MacroDecl 0x11c03a0 <line:5:9> col:9 M1 defined
>>>
>>> `-MacroDecl 0x11c03f8 <mainfile.c:3:9> col:9 M2 defined
>>>
>>> TranslationUnitDecl 0xd6c078 <<invalid sloc>>
<invalid sloc>
>>>
>>> |-TypedefDecl 0xd6c330 <<invalid sloc>> <invalid
sloc> implicit
>>> __int128_t '__int128'
>>>
>>> |-TypedefDecl 0xd6c370 <<invalid sloc>> <invalid
sloc> implicit
>>> __uint128_t 'unsigned __int128'
>>>
>>> |-TypedefDecl 0xd6c3c8 <<invalid sloc>> <invalid
sloc> implicit
>>> __builtin_ms_va_list 'char *'
>>>
>>> `-TypedefDecl 0xd6c590 <<invalid sloc>> <invalid
sloc> implicit
>>> __builtin_va_list 'struct __va_list_tag [1]'
>>>
>>>
========================================================>>>
>>>
>>>
>>>
>>>
>>> LLVM IR
>>>
>>>
========================================================>>>
>>> target datalayout =
"e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>>
>>> target triple = "x86_64-pc-linux"
>>>
>>>
>>>
>>> !llvm.dbg.cu = !{!0}
>>>
>>> !llvm.module.flags = !{!327}
>>>
>>> !llvm.ident = !{!328}
>>>
>>>
>>>
>>> !0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1,
producer:
>>> "clang version 3.8.0 (trunk 251321)", isOptimized: false,
runtimeVersion:
>>> 0, emissionKind: 1, enums: !2, macros: !3)
>>>
>>> !1 = !DIFile(filename: "mainfile.c", directory:
"/")
>>>
>>> !2 = !{}
>>>
>>> !3 = !{!4, !5, … <More compiler macros> … , !312, !313}
>>>
>>> !4 = !DIMacro(macro type: DW_MACINFO_define, name:
"__llvm__", value:
>>> !"1")
>>>
>>> !5 = !DIMacro(macro type: DW_MACINFO_define, name:
"__clang__", value:
>>> !"1")
>>>
>>>
>>>
>>> … <More compiler macros> …
>>>
>>>
>>>
>>> !312 = !DIMacro(macro type: DW_MACINFO_define, name:
"M3", value:
>>> !"Value3")
>>>
>>> !313 = !DIFileInclude(file: !314, nodes: !315)
>>>
>>> !314 = !DIFile(filename: "mainfile.c", directory:
"/")
>>>
>>> !315 = !{!316, !320, !321, !326}
>>>
>>> !316 = !DIFileInclude(file: !317, nodes: !318)
>>>
>>> !317 = !DIFile(filename: "myfile2.h", directory:
"/")
>>>
>>> !318 = !{!319}
>>>
>>> !319 = !DIMacro(macro type: DW_MACINFO_define, name:
"M4", value:
>>> !"Value4")
>>>
>>> !320 = !DIMacro(macro type: DW_MACINFO_define, name:
"M1", line: 1,
>>> value: !"Value1")
>>>
>>> !321 = !DIFileInclude(line: 2, file: !322, nodes: !323)
>>>
>>> !322 = !DIFile(filename: "myfile.h", directory:
"/")
>>>
>>> !323 = !{!324, !325}
>>>
>>> !324 = !DIMacro(macro type: DW_MACINFO_undef, name: "M1",
line: 4)
>>>
>>> !325 = !DIMacro(macro type: DW_MACINFO_define, name:
"M1", line: 5,
>>> value: !"NewValue1")
>>>
>>> !326 = !DIMacro(macro type: DW_MACINFO_define, name:
"M2(x,y)", line: 3,
>>> value: !"( (x) + (y) * Value2)")
>>>
>>> !327 = !{i32 2, !"Debug Info Version", i32 3}
>>>
>>> !328 = !{!"clang version 3.8.0 (trunk 251321)"}
>>>
>>>
========================================================>>>
>>>
>>>
>>>
>>>
>>> DWARF
>>>
>>>
========================================================>>>
>>> Command line: llvm-dwarfdump.exe -debug-dump=macro mainfile.o
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>> mainfile3.o:  file format ELF64-x86-64
>>>
>>>
>>>
>>> .debug_macinfo contents:
>>>
>>> DW_MACINFO_define - lineno: 0 macro: __llvm__ 1
>>>
>>> DW_MACINFO_define - lineno: 0 macro: __clang__ 1
>>>
>>>
>>>
>>> … <More compiler macros> …
>>>
>>>
>>>
>>> DW_MACINFO_define - lineno: 0 macro: M3 Value3
>>>
>>> DW_MACINFO_start_file - lineno: 0 filenum: 1
>>>
>>>   DW_MACINFO_start_file - lineno: 0 filenum: 2
>>>
>>>     DW_MACINFO_define - lineno: 0 macro: M4 Value4
>>>
>>>   DW_MACINFO_end_file
>>>
>>>   DW_MACINFO_define - lineno: 1 macro: M1 Value1
>>>
>>>   DW_MACINFO_start_file - lineno: 2 filenum: 3
>>>
>>>     DW_MACINFO_undef - lineno: 4 macro: M1
>>>
>>>     DW_MACINFO_define - lineno: 5 macro: M1 NewValue1
>>>
>>>   DW_MACINFO_end_file
>>>
>>>   DW_MACINFO_define - lineno: 3 macro: M2(x,y) ( (x) + (y) *
Value2)
>>>
>>> DW_MACINFO_end_file
>>>
>>>
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>> Command line: llvm-dwarfdump.exe -debug-dump=line mainfile.o
>>>
>>>
>>>
--------------------------------------------------------------------------------------
>>>
>>> .debug_line contents:
>>>
>>>
>>>
>>> … <Other line table Info> …
>>>
>>>
>>>
>>>                 Dir  Mod Time   File Len   File Name
>>>
>>>                 ---- ---------- ----------
---------------------------
>>>
>>> file_names[  1]    1 0x00000000 0x00000000 mainfile.c
>>>
>>> file_names[  2]    1 0x00000000 0x00000000 myfile2.h
>>>
>>> file_names[  3]    1 0x00000000 0x00000000 myfile.h
>>>
>>>
========================================================>>>
>>>
>>>
>>>
---------------------------------------------------------------------
>>> Intel Israel (74) Limited
>>>
>>> This e-mail and any attachments may contain confidential material
for
>>> the sole use of the intended recipient(s). Any review or
distribution
>>> by others is strictly prohibited. If you are not the intended
>>> recipient, please contact the sender and delete all copies.
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>>
>>>
---------------------------------------------------------------------
>>> Intel Israel (74) Limited
>>>
>>> This e-mail and any attachments may contain confidential material
for
>>> the sole use of the intended recipient(s). Any review or
distribution
>>> by others is strictly prohibited. If you are not the intended
>>> recipient, please contact the sender and delete all copies.
>>>
>>>
>>>
>>>
---------------------------------------------------------------------
>>> Intel Israel (74) Limited
>>>
>>> This e-mail and any attachments may contain confidential material
for
>>> the sole use of the intended recipient(s). Any review or
distribution
>>> by others is strictly prohibited. If you are not the intended
>>> recipient, please contact the sender and delete all copies.
>>>
>>>
---------------------------------------------------------------------
>>> Intel Israel (74) Limited
>>>
>>> This e-mail and any attachments may contain confidential material
for
>>> the sole use of the intended recipient(s). Any review or
distribution
>>> by others is strictly prohibited. If you are not the intended
>>> recipient, please contact the sender and delete all copies.
>>>
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151113/90eef5a6/attachment-0001.html>