llvm dev - Aug 2017 - Extending TableGen's 'foreach' to work with 'multiclass' and 'defm'

I have been reading the “RFC/bikeshedding: Separation of instruction and pattern
definitions in LLVM backends” topic with considerable interest.  This is an
approach I have been considering for taming our own large instruction set, and
it looks like it structures our descriptions better than the conventional
approach we have used so far.

 

However, I have another form of TableGen taming that I would like to do.

 

In addition to the separation of instruction from the patterns that use it, I
have also got a large number of “instruction groups” that differ in their
schedules and operands, but are in all other respects structurally similar.

 

For example, I have a large number of load instructions that are almost
identical but which are in 5 specific groups:

 

*         Loads where the memory operand is in a register [LDA]

*         Loads where the memory operand is in a register and is
auto-incremented by an implicit value [LDB]

*         Loads where the memory operand is in a register and is
auto-incremented by a value in another register [LDC]

*         Loads where the memory operand has a base pointer in a register and an
immediate offset [LDD]

*         Loads where the memory operand has a base pointer in a register and an
offset in a register [LDE]

 

If I don’t have the multiple processor versions, I can use ‘class/def’ by
specifying the appropriate ‘class’ for each type of instruction in the group,
and then use a common set of ‘def’ declarations for them with ‘foreach’, for
example:

 

// Describe the meta-classes for the LDA group

class T_LDA_Type1 : ...

class T_LDA_Type2 : ...

 

// Describe the meta-classes for the LDB group

class T_LDB_Type1 : ...

class T_LDB_Type2 : ...

 

// Describe the meta-classes for the LDC group

class T_LDC_Type1 : ...

class T_LDC_Type2 : ...

 

// Describe the meta-classes for the LDD group

class T_LDD_Type1 : ...

class T_LDD_Type2 : ...

 

// Describe the meta-classes for the LDE group

class T_LDE_Type1 : ...

class T_LDE_Type2 : ...

 

// Share a single set of definitions, but parameterised by meta-class

foreach loadOp = ["LDA", "LDB", "LDC",
"LDD", "LDE" ] in {

  def Prefix_#loadOp#_suffix1 : T_#loadOp#_Type1<...>;

  def Prefix_#loadOp#_suffix2 : T_#loadOp#_Type2<...>;

}

 

All of the ‘def’s pass the same values to the ‘class’s, though the ‘class’s may
ignore some as appropriate.  For example, I pass the auto-increment size to
each, though only the auto-increment patterns care.

 

This neatly allows me to symmetrically manage all the instructions in each of
the groups using a single statement of the patterns, and maintain only one fifth
of the number of definitions.  In my actual source, there are around 50
different types of instruction within each group, so reducing the repetition is
quite significant.

 

But there is a downside.

 

For each of the above I also have variations that are a result of different
processor and ISA versions, and because of this I have to use ‘multiclass/defm’
to define the descriptions along with ‘Require’ predicates.  The same approach
does not work with ‘multiclass/defm’ though, because TableGen does not support
‘foreach’ with ‘multiclass/defm’.

 

I have experimented with adapting TableGen to do this, but I am just not
knowledgeable enough about how TableGen works and my attempts have not been
successful.  Perhaps some of the people debating the separation of instruction
and patterns topic might have some insight into how TableGen might be adapted to
support ‘foreach’ with ‘multiclass/defm’ definitions and could advise me how I
should do this; or maybe the maintainers of TableGen might consider this
something that they would be willing to add to TableGen in the future?

 

Thanks,

 

            MartinO

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170821/88a774ce/attachment.html>

On 21 August 2017 at 13:23, Martin J. O'Riordan via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> But there is a downside.
>
> For each of the above I also have variations that are a result of different
> processor and ISA versions, and because of this I have to use
> ‘multiclass/defm’ to define the descriptions along with ‘Require’
> predicates.  The same approach does not work with ‘multiclass/defm’ though,
> because TableGen does not support ‘foreach’ with ‘multiclass/defm’.
>
> I have experimented with adapting TableGen to do this, but I am just not
> knowledgeable enough about how TableGen works and my attempts have not been
> successful.  Perhaps some of the people debating the separation of
> instruction and patterns topic might have some insight into how TableGen
> might be adapted to support ‘foreach’ with ‘multiclass/defm’ definitions
and
> could advise me how I should do this; or maybe the maintainers of TableGen
> might consider this something that they would be willing to add to TableGen
> in the future?
Hi Martin, I think this is an interesting topic. I've also run up
against the limitations of foreach, though for my particular case the
variable-sized register class work provides a better solution.

I will note that at least one backend (Hexagon) has moved towards
using TableGen as a fairly 'dumb' data definition language, relying on
a separate tool for generating instruction definitions. I'd be curious
to know if others are using this approach. It'd also imaging that
using m4/jinja or similar as a .td preprocessor would be a potential
option for an out-of-tree backend, in cases where TableGen macro
support and programmability is too weak.

I suppose one question is: would allowing foreach to be used with
multiclass/defm be sufficient to allow TableGen to be a productive and
maintainable way of defining complex architectures, or would there be
a number of other deficiencies that might push you towards larger
TableGen extensions or using a separate tool or preprocessor?

Best,

Alex

On 08/22/2017 03:59 AM, Alex Bradbury via llvm-dev
wrote:
> On 21 August 2017 at 13:23, Martin J. O'Riordan via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> But there is a downside.
>>
>> For each of the above I also have variations that are a result of
different
>> processor and ISA versions, and because of this I have to use
>> ‘multiclass/defm’ to define the descriptions along with ‘Require’
>> predicates.  The same approach does not work with ‘multiclass/defm’
though,
>> because TableGen does not support ‘foreach’ with ‘multiclass/defm’.
>>
>> I have experimented with adapting TableGen to do this, but I am just
not
>> knowledgeable enough about how TableGen works and my attempts have not
been
>> successful.  Perhaps some of the people debating the separation of
>> instruction and patterns topic might have some insight into how
TableGen
>> might be adapted to support ‘foreach’ with ‘multiclass/defm’
definitions and
>> could advise me how I should do this; or maybe the maintainers of
TableGen
>> might consider this something that they would be willing to add to
TableGen
>> in the future?
> Hi Martin, I think this is an interesting topic. I've also run up
> against the limitations of foreach, though for my particular case the
> variable-sized register class work provides a better solution.
>
> I will note that at least one backend (Hexagon) has moved towards
> using TableGen as a fairly 'dumb' data definition language, relying
on
> a separate tool for generating instruction definitions. I'd be curious
> to know if others are using this approach. It'd also imaging that
> using m4/jinja or similar as a .td preprocessor would be a potential
> option for an out-of-tree backend, in cases where TableGen macro
> support and programmability is too weak.
>
> I suppose one question is: would allowing foreach to be used with
> multiclass/defm be sufficient to allow TableGen to be a productive and
> maintainable way of defining complex architectures, or would there be
> a number of other deficiencies that might push you towards larger
> TableGen extensions or using a separate tool or preprocessor?
The fact that you can't use foreach with multiclasses is a bug, and we 
should fix it, if possible, regardless of whether it is the last 
remaining roadblock to handling complex architectures. For situations 
well beyond TableGen's current language capabilities, we have a decision 
to make. We can continue extending TableGen until it can meet those 
needs. Alternatively, we can enable the use of some more-powerful input 
language. For example, we could allow TableGen to embed Python, and then 
use Python in order to generate record definitions.

  -Hal
>
> Best,
>
> Alex
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory