llvm dev - Aug 2020 - TableGen trace facility

Are all the records collected as they are parsed, with template parameter
substitution and lets, and *then*, after all records are collected, a
"pass" is made to calculate the inter-field expressions?

Once I understand this, I will add a section to the new guide to explain it. I
presume it is the case that this behavior should be publicized.

It also appears to be the case that a record is created and bound to its name
before fields are inherited from its superclasses, which is why you can write:

class A <dag d> {
  dag the_dag = d;
}

def rec1 : A<(ops rec1)>

Do I understand that correctly?

At 8/4/2020 05:38 PM, Nicolai Hähnle wrote:
>On Tue, Aug 4, 2020 at 11:13 PM Paul C. Anagnostopoulos
><paul at windfall.com> wrote:
>> Yes, I understand the problem. To be more useful, TableGen would have
to carry the traces along with the classes and records and (re)display the
values while the substitutions are being made.
>>
>> I'm writing a new Programmer's Guide for TableGen and have been
digging into the parse-time versus substitution-time issue. I haven't found
a document that makes it clear. Can you give a quick summary of the phases?
>
>There aren't really any phases per se, in the sense that the TableGen
>frontend doesn't have passes. Maybe that's a mistake, but that's
how
>it evolved.
>
>So instead, every time a record is "instantiated" -- whether
that's by
>defining a new class that derives from one or more pre-existing
>classes, or by a def in a multi-class, or by a free-standing def, or a
>defm (inside or outside a multiclass) -- any free variables that are
>"resolved" as template parameters get their substitution applied.
The
>relevant let-statements take effect just after that.
>
>However, and this is key, reference _between_ fields are only resolved
>and substituted once the final defined record at the global scope is
>produced. This is what makes:
>
>class A<int x> {
>  int a = x;
>  int b = a + 1;
>}
>
>def B : A<5> { let a = 10; }
>
>... result in b == 11 instead of b == 6.
>
>There's probably subtlety here that I'm forgetting, but that's
the
>very short version of it ;)
>
>Cheers,
>Nicolai
>
>
>>
>> At 8/4/2020 05:01 PM, Nicolai Hähnle wrote:
>> >On Mon, Aug 3, 2020 at 6:04 PM Paul C. Anagnostopoulos via llvm-dev
>> ><llvm-dev at lists.llvm.org> wrote:
>> >> A question for those of you who have developed complex
TableGen files: Do you think a trace facility would be useful during debugging?
The idea is to add a new statement to TableGen along these lines:
>> >>
>> >>   trace tag : value1, value2, ... ;
>> >>
>> >> When encountered, the TableGen parser would display the tag
along with the values of the specified value1, value2, etc. The tag is an
identifier that makes it easier to distinguish multiple traces.
>> >
>> >I'm not so sure that that's particularly useful or how it
would even
>> >work. The issue is that at the point in time where the frontend
parses
>> >those trace tags, most substitutions haven't taken place yet,
so
>> >you'll usually get fairly trivial answers that by themselves
aren't
>> >particularly helpful.
>> >
>> >Some form of inspection of how values are substituted would indeed
be
>> >helpful, I just don't think the "trace" is quite it.
In a perfect
>> >world, we'd have some sort of "record database
explorer" that doesn't
>> >just let you look at all the final records (TableGen already allows
>> >you to do that), but also allows you to interactively explore their
>> >"history", i.e. how did the records come to be.
>> >
>> >Cheers,
>> >Nicolai
>> >
>> >--
>> >Lerne, wie die Welt wirklich ist,
>> >aber vergiss niemals, wie sie sein sollte.

On Wed, Aug 5, 2020 at 1:08 AM Paul C. Anagnostopoulos
<paul at windfall.com> wrote:
> Are all the records collected as they are parsed, with template parameter
substitution and lets, and *then*, after all records are collected, a
"pass" is made to calculate the inter-field expressions?
No, the inter-field expressions are resolved immediately as the final
"def" is processed. See TGParser::addDefOne.

> Once I understand this, I will add a section to the new guide to explain
it. I presume it is the case that this behavior should be publicized.
>
> It also appears to be the case that a record is created and bound to its
name before fields are inherited from its superclasses, which is why you can
write:
>
> class A <dag d> {
>   dag the_dag = d;
> }
>
> def rec1 : A<(ops rec1)>
>
> Do I understand that correctly?
Almost :)

In reality, this is allowed as a special-case by
TGParser::ParseIDValue, whose purpose is looking up identifiers. See
the comment about self-references towards the bottom of the method.

Cheers,
Nicolai

>
> At 8/4/2020 05:38 PM, Nicolai Hähnle wrote:
> >On Tue, Aug 4, 2020 at 11:13 PM Paul C. Anagnostopoulos
> ><paul at windfall.com> wrote:
> >> Yes, I understand the problem. To be more useful, TableGen would
have to carry the traces along with the classes and records and (re)display the
values while the substitutions are being made.
> >>
> >> I'm writing a new Programmer's Guide for TableGen and have
been digging into the parse-time versus substitution-time issue. I haven't
found a document that makes it clear. Can you give a quick summary of the
phases?
> >
> >There aren't really any phases per se, in the sense that the
TableGen
> >frontend doesn't have passes. Maybe that's a mistake, but
that's how
> >it evolved.
> >
> >So instead, every time a record is "instantiated" -- whether
that's by
> >defining a new class that derives from one or more pre-existing
> >classes, or by a def in a multi-class, or by a free-standing def, or a
> >defm (inside or outside a multiclass) -- any free variables that are
> >"resolved" as template parameters get their substitution
applied. The
> >relevant let-statements take effect just after that.
> >
> >However, and this is key, reference _between_ fields are only resolved
> >and substituted once the final defined record at the global scope is
> >produced. This is what makes:
> >
> >class A<int x> {
> >  int a = x;
> >  int b = a + 1;
> >}
> >
> >def B : A<5> { let a = 10; }
> >
> >... result in b == 11 instead of b == 6.
> >
> >There's probably subtlety here that I'm forgetting, but
that's the
> >very short version of it ;)
> >
> >Cheers,
> >Nicolai
> >
> >
> >>
> >> At 8/4/2020 05:01 PM, Nicolai Hähnle wrote:
> >> >On Mon, Aug 3, 2020 at 6:04 PM Paul C. Anagnostopoulos via
llvm-dev
> >> ><llvm-dev at lists.llvm.org> wrote:
> >> >> A question for those of you who have developed complex
TableGen files: Do you think a trace facility would be useful during debugging?
The idea is to add a new statement to TableGen along these lines:
> >> >>
> >> >>   trace tag : value1, value2, ... ;
> >> >>
> >> >> When encountered, the TableGen parser would display the
tag along with the values of the specified value1, value2, etc. The tag is an
identifier that makes it easier to distinguish multiple traces.
> >> >
> >> >I'm not so sure that that's particularly useful or how
it would even
> >> >work. The issue is that at the point in time where the
frontend parses
> >> >those trace tags, most substitutions haven't taken place
yet, so
> >> >you'll usually get fairly trivial answers that by
themselves aren't
> >> >particularly helpful.
> >> >
> >> >Some form of inspection of how values are substituted would
indeed be
> >> >helpful, I just don't think the "trace" is quite
it. In a perfect
> >> >world, we'd have some sort of "record database
explorer" that doesn't
> >> >just let you look at all the final records (TableGen already
allows
> >> >you to do that), but also allows you to interactively explore
their
> >> >"history", i.e. how did the records come to be.
> >> >
> >> >Cheers,
> >> >Nicolai
> >> >
> >> >--
> >> >Lerne, wie die Welt wirklich ist,
> >> >aber vergiss niemals, wie sie sein sollte.
>

-- 
Lerne, wie die Welt wirklich ist,
aber vergiss niemals, wie sie sein sollte.

Hi Paul,
If all you care about is debugging then for now we can just emit a few more
debug messages which would help to "trace" the flow. To distinguish
traces
you can prefix it with some known string. I don't think you really need a
'trace' tag in the language spec for this.

Debugging Tablegen has always been a nightmare and I don't think we can
ever reach a stage where we can start a debugger and debug statements in
.td files step-by-step. This is far from reality unless we fundamentally
change the language.



On Wed, Aug 5, 2020 at 1:32 PM Nicolai Hähnle via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Wed, Aug 5, 2020 at 1:08 AM Paul C. Anagnostopoulos
> <paul at windfall.com> wrote:
> > Are all the records collected as they are parsed, with template
> parameter substitution and lets, and *then*, after all records are
> collected, a "pass" is made to calculate the inter-field
expressions?
>
> No, the inter-field expressions are resolved immediately as the final
> "def" is processed. See TGParser::addDefOne.
>
>
> > Once I understand this, I will add a section to the new guide to
explain
> it. I presume it is the case that this behavior should be publicized.
> >
> > It also appears to be the case that a record is created and bound to
its
> name before fields are inherited from its superclasses, which is why you
> can write:
> >
> > class A <dag d> {
> >   dag the_dag = d;
> > }
> >
> > def rec1 : A<(ops rec1)>
> >
> > Do I understand that correctly?
>
> Almost :)
>
> In reality, this is allowed as a special-case by
> TGParser::ParseIDValue, whose purpose is looking up identifiers. See
> the comment about self-references towards the bottom of the method.
>
> Cheers,
> Nicolai
>
>
> >
> > At 8/4/2020 05:38 PM, Nicolai Hähnle wrote:
> > >On Tue, Aug 4, 2020 at 11:13 PM Paul C. Anagnostopoulos
> > ><paul at windfall.com> wrote:
> > >> Yes, I understand the problem. To be more useful, TableGen
would have
> to carry the traces along with the classes and records and (re)display the
> values while the substitutions are being made.
> > >>
> > >> I'm writing a new Programmer's Guide for TableGen and
have been
> digging into the parse-time versus substitution-time issue. I haven't
found
> a document that makes it clear. Can you give a quick summary of the phases?
> > >
> > >There aren't really any phases per se, in the sense that the
TableGen
> > >frontend doesn't have passes. Maybe that's a mistake, but
that's how
> > >it evolved.
> > >
> > >So instead, every time a record is "instantiated" --
whether that's by
> > >defining a new class that derives from one or more pre-existing
> > >classes, or by a def in a multi-class, or by a free-standing def,
or a
> > >defm (inside or outside a multiclass) -- any free variables that
are
> > >"resolved" as template parameters get their substitution
applied. The
> > >relevant let-statements take effect just after that.
> > >
> > >However, and this is key, reference _between_ fields are only
resolved
> > >and substituted once the final defined record at the global scope
is
> > >produced. This is what makes:
> > >
> > >class A<int x> {
> > >  int a = x;
> > >  int b = a + 1;
> > >}
> > >
> > >def B : A<5> { let a = 10; }
> > >
> > >... result in b == 11 instead of b == 6.
> > >
> > >There's probably subtlety here that I'm forgetting, but
that's the
> > >very short version of it ;)
> > >
> > >Cheers,
> > >Nicolai
> > >
> > >
> > >>
> > >> At 8/4/2020 05:01 PM, Nicolai Hähnle wrote:
> > >> >On Mon, Aug 3, 2020 at 6:04 PM Paul C. Anagnostopoulos
via llvm-dev
> > >> ><llvm-dev at lists.llvm.org> wrote:
> > >> >> A question for those of you who have developed
complex TableGen
> files: Do you think a trace facility would be useful during debugging? The
> idea is to add a new statement to TableGen along these lines:
> > >> >>
> > >> >>   trace tag : value1, value2, ... ;
> > >> >>
> > >> >> When encountered, the TableGen parser would display
the tag along
> with the values of the specified value1, value2, etc. The tag is an
> identifier that makes it easier to distinguish multiple traces.
> > >> >
> > >> >I'm not so sure that that's particularly useful
or how it would even
> > >> >work. The issue is that at the point in time where the
frontend
> parses
> > >> >those trace tags, most substitutions haven't taken
place yet, so
> > >> >you'll usually get fairly trivial answers that by
themselves aren't
> > >> >particularly helpful.
> > >> >
> > >> >Some form of inspection of how values are substituted
would indeed be
> > >> >helpful, I just don't think the "trace" is
quite it. In a perfect
> > >> >world, we'd have some sort of "record database
explorer" that doesn't
> > >> >just let you look at all the final records (TableGen
already allows
> > >> >you to do that), but also allows you to interactively
explore their
> > >> >"history", i.e. how did the records come to be.
> > >> >
> > >> >Cheers,
> > >> >Nicolai
> > >> >
> > >> >--
> > >> >Lerne, wie die Welt wirklich ist,
> > >> >aber vergiss niemals, wie sie sein sollte.
> >
>
>
> --
> Lerne, wie die Welt wirklich ist,
> aber vergiss niemals, wie sie sein sollte.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

-- 
*Disclaimer: Views, concerns, thoughts, questions, ideas expressed in this
mail are of my own and my employer has no take in it. *
Thank You.
Madhur D. Amilkanthwar
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200805/92cb8870/attachment.html>