llvm dev - Jun 2018 - [lldb-dev] Adding DWARF5 accelerator table support to llvm

How do you handle name lookup for nested classes? They have the same
problem (they don't appear in all definitions) - don't appear in all
descriptions of the outer/parent class. (in theory we could ensure there's
always at least a declaration of the nested class - but we don't even do
that if the nested class is unused)

Is it just the case that Clang doesn't mind you adding a new nested class
but it does mind you adding a new member function template? If so, maybe we
could change Clang to support adding new member function templates instead
of extending DWARF?

On Fri, Jun 15, 2018 at 11:59 AM <paul.robinson at sony.com> wrote:
> gc> Solution #1 would cause us to dig through all definitions of all C++
> gc> classes all the time when parsing DWARF to check if definitions of
> gc> the classes had template methods. And we would need to find the
class
> gc> that has the most template methods. This would cause us to parse
much
> gc> more of the debug info all of the time and cause increased memory
> gc> consumption and performance regressions.
>
> pr> It would be cheap to put a flag on the class DIE that tells you
there
> pr> are template methods to go look for.  Then you incur the cost only
> pr> when necessary.  And the accelerator table makes it fast to find the
> pr> other class descriptions.
>
> gc> That is a fine solution. But we still run into the problem where we
> don't
> gc> know if the DWARF knows about that flag. If we do a flag, it would
be
> nice
> gc> if it were mandatory on all classes to indicate support for the
flag.
> But
> gc> this would be a fine solution and not hard to implement.
>
> pr> So what you really want is not a flag, but a count, so you can tell
> when
> pr> you've found all the different templates.  If the count is zero,
> there's
> pr> nothing to look for.  If the count is two, you look around at all
the
> pr> various definitions of the class until you find two different
> templates,
> pr> then you stop.  If there's no count attribute, your producer
doesn't
> pr> know you want this information and you do it the hard way.  Or,
we've
> pr> invented a way to describe the templates directly in the class.
> pr>
> pr> How's that?
>
> gc> that would work fine.
>
> I filed PR37816 to track this idea.
>
> The other ideas:
>
>  - accelerator to point to the actual instantiations
>  - emitting template definitions not just instantiations
>
> would be trickier to define and harder to implement correctly.
> I won't say they can't be done, but somebody else would have to do
> the heavy lifting here, unless it turns out that our debugger folks
> like the idea.
>
> --paulr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180615/d978dfe0/attachment.html>

On Fri, 15 Jun 2018 at 20:14, David Blaikie <dblaikie at gmail.com>
wrote:
>
> How do you handle name lookup for nested classes? They have the same
problem (they don't appear in all definitions) - don't appear in all
descriptions of the outer/parent class. (in theory we could ensure there's
always at least a declaration of the nested class - but we don't even do
that if the nested class is unused)
>
> Is it just the case that Clang doesn't mind you adding a new nested
class but it does mind you adding a new member function template? If so, maybe
we could change Clang to support adding new member function templates instead of
extending DWARF?

I was thinking about the same thing. It seems to me that this could be
viewed as a deficiency of our implementation of dwarf parsing. (It's a
pretty understandable deficiency, given that we are based on clang
(compiler), and it thinks of the types in the same way as C++ does --
incomplete; or complete with template members and all). However, these
template member functions should not impact anything "important" in
the class (data member layout, vtables, ...) so one could conceivably
have an implementation which allows member addition on the fly. And in
this case the existing accelerator tables would work perfectly -- we
would get a query "does this class have method X", we would look at
the accel table, and it would point us straight to X. However, I have
no idea how hard would it be to fit this scheme into the existing
clang/lldb design.



That said, having DWARF be able to represent the template member
functions in an abstract way also sounds like nice thing to have from
a debug info format.

pl

> On Jun 15, 2018, at 1:48 PM, Pavel Labath <labath at google.com>
wrote:
> 
> On Fri, 15 Jun 2018 at 20:14, David Blaikie <dblaikie at gmail.com>
wrote:
>> 
>> How do you handle name lookup for nested classes? They have the same
problem (they don't appear in all definitions) - don't appear in all
descriptions of the outer/parent class. (in theory we could ensure there's
always at least a declaration of the nested class - but we don't even do
that if the nested class is unused)
>> 
>> Is it just the case that Clang doesn't mind you adding a new nested
class but it does mind you adding a new member function template? If so, maybe
we could change Clang to support adding new member function templates instead of
extending DWARF?
> 
> 
> I was thinking about the same thing. It seems to me that this could be
> viewed as a deficiency of our implementation of dwarf parsing. (It's a
> pretty understandable deficiency, given that we are based on clang
> (compiler), and it thinks of the types in the same way as C++ does --
> incomplete; or complete with template members and all). However, these
> template member functions should not impact anything "important"
in
> the class (data member layout, vtables, ...) so one could conceivably
> have an implementation which allows member addition on the fly. And in
> this case the existing accelerator tables would work perfectly -- we
> would get a query "does this class have method X", we would look
at
> the accel table, and it would point us straight to X. However, I have
> no idea how hard would it be to fit this scheme into the existing
> clang/lldb design.
For a little background on how the DWARF parsing works. When anything has a type
"class A" in LLDB (variable, method in class A, etc), we first create
a forward declaration to class A in the clang AST. Each symbol file installs
itself as an external AST source that can complete any types when requested.
This uses the functionality that mimics each symbol file being a precompiled
header. Since we have an external AST source, clang can ask for the type to be
completed at any time (just like clang does with PCH files), which will call
back into the external AST source and cause the type to be completed.

In LLDB, we have 3 distinct states of a type: forward declaration to type,
completed type, the type we need for layout. If a variable has a type
"class A*", then when displaying this variable in the output of
"frame var", we don't need to know the full definition of class A.
If the user expands a type in say an IDE variable view or types an expression
that results in a class A instance or reference, we will get the complete
version of the type. When we are making other types that include class A through
inheritance or as a member variable, we may or may not need get the full
definition for class A. For example if we have a "class A* m_a_ptr;"
member variable, we don't need to know about A since we only need to know
what type we need in order to layout the type which is a pointer here. If we
inherit from class A or have a "class A m_a;" then the layout type
needed here would be the full type. So we are very careful to only expand the
type when and if needed in LLDB.

That being said we currently only have two versions of the type as far as clang
know: forward and complete. I think this is how clang expects things to be as
well. So the above approach could work as long as we can teach clang to ask the
external AST source about addition names when doing method lookups, but that
will take some clang modifications that will probably only need to be enabled in
debugger mode. Also if we did add this ability, we will want to limit it to
classes that have templated functions we don't want to always ask a class we
have completed to continually look for things when the user types an expression
wrong like "a.this_method_never_will_exist_in_a()". Maybe that is ok
though.

If we are able to make the above approach work, then there will need to be an
additional fix where we would need to teach AST importer to be able to compare
two classes and ignore any template methods when doing the compare when in
debugger mode only. We might have two shared libraries that each have a class A,
one with one specialized function and one with another, and when one or both of
those get imported into the expression or target AST, they would need to merge
the type correctly when importing it. Right now if there are two class trying be
to imported at the same decl context layer the import will fail if the class
exists at the same level and isn't exactly the same.

So quite a bit of clang work if we want to make this work.

Each symbol file installs itself as an external AST source so if we wanted to
add the template member functions on the fly, hopefully we would get a request
that says in the DeclContext "class A" what is "Foo" if you
type something like:

A a;
(lldb) a.Foo<int>()

But I am not sure what clang will do if it already thinks it knows the
definition for class A, so we might need to teach clang, that in debugger mode
only, check the external AST source if there is one to see.

> 
> 
> That said, having DWARF be able to represent the template member
> functions in an abstract way also sounds like nice thing to have from
> a debug info format.
Yes, that would be great, but will require DWARF changes and is much more long
term.