llvm dev - Aug 2017 - Extending TableGen's 'foreach' to work with 'multiclass' and 'defm'

On 08/23/2017 11:35 AM, Jakob Stoklund Olesen wrote:
>
>> On Aug 22, 2017, at 14:15, Hal Finkel via llvm-dev 
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>>
>> For situations well beyond TableGen's current language
capabilities,
>> we have a decision to make. We can continue extending TableGen until 
>> it can meet those needs. Alternatively, we can enable the use of some 
>> more-powerful input language. For example, we could allow TableGen to 
>> embed Python, and then use Python in order to generate record 
>> definitions.
>
> For a project that’s not LLVM, I recently had the opportunity to 
> replace both TableGen and *.td files with Python scripts. I found that 
> TableGen’s features were easily matched by Python’s for loops and the 
> ability to define functions. I am pretty happy with the approach so 
> far. AMA
Interesting, thanks.
>
> This is a lot easier to do in a green field project than in an old 
> project like LLVM, of course.
If we want to go down that route, I can certainly imagine a feasible 
incremental-transitioning strategy. We could allow TableGen to use an 
embedded Python interpreter to generate records based on Python data 
structures, and then, combine records from the existing .td files with 
those generated by the Python code. We'd use the existing TableGen 
plugins (which we may need to continue to use regardless, compared to 
writing Python, for performance reasons), and so we could incrementally 
transition existing definitions from .td files to Python as appropriate.

  -Hal
>
> Example “.td” file: 
>
https://github.com/stoklund/cretonne/blob/master/lib/cretonne/meta/isa/riscv/encodings.py
>
> Thanks,
> /jakob
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170823/925a7c33/attachment.html>

On 8/23/2017 11:58 AM, Hal Finkel via llvm-dev wrote:
> If we want to go down that route, I can certainly imagine a feasible 
> incremental-transitioning strategy. We could allow TableGen to use an 
> embedded Python interpreter to generate records based on Python data 
> structures, and then, combine records from the existing .td files with 
> those generated by the Python code. We'd use the existing TableGen 
> plugins (which we may need to continue to use regardless, compared to 
> writing Python, for performance reasons), and so we could incrementally 
> transition existing definitions from .td files to Python as appropriate.
Would we then eliminate TableGen completely in the long term?

-Krzysztof

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, 
hosted by The Linux Foundation

On 08/23/2017 12:06 PM, Krzysztof Parzyszek via llvm-dev
wrote:
> On 8/23/2017 11:58 AM, Hal Finkel via llvm-dev wrote:
>> If we want to go down that route, I can certainly imagine a feasible 
>> incremental-transitioning strategy. We could allow TableGen to use an 
>> embedded Python interpreter to generate records based on Python data 
>> structures, and then, combine records from the existing .td files 
>> with those generated by the Python code. We'd use the existing 
>> TableGen plugins (which we may need to continue to use regardless, 
>> compared to writing Python, for performance reasons), and so we could 
>> incrementally transition existing definitions from .td files to 
>> Python as appropriate.
>
> Would we then eliminate TableGen completely in the long term?
That could also be two separate questions: Would we replace the .td 
input language with Python completely in the long term? Would we rewrite 
the the backends (i.e., TableGen plugins) in Python? I don't yet have an 
opinion on either. I can see advantages to providing Python as input 
language. What do you think?

  -Hal
>
> -Krzysztof
>
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

On 08/23/2017 09:58 AM, Hal Finkel via llvm-dev wrote:
> 
> On 08/23/2017 11:35 AM, Jakob Stoklund Olesen wrote:
>>
>>> On Aug 22, 2017, at 14:15, Hal Finkel via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>
>>> For situations well beyond TableGen's current language
capabilities, we have a decision to make. We can continue extending TableGen
until it can meet those needs. Alternatively, we can enable the use of some
more-powerful input language. For example, we could allow TableGen to embed
Python, and then use Python in order to generate record definitions.
>>
>> For a project that’s not LLVM, I recently had the opportunity to
replace both TableGen and *.td files with Python scripts. I found that
TableGen’s features were easily matched by Python’s for loops and the ability to
define functions. I am pretty happy with the approach so far. AMA
> 
> Interesting, thanks.
> 
>>
>> This is a lot easier to do in a green field project than in an old
project like LLVM, of course.
> 
> If we want to go down that route, I can certainly imagine a feasible
incremental-transitioning strategy. We could allow TableGen to use an embedded
Python interpreter to generate records based on Python data structures, and
then, combine records from the existing .td files with those generated by the
Python code. We'd use the existing TableGen plugins (which we may need to
continue to use regardless, compared to writing Python, for performance
reasons), and so we could incrementally transition existing definitions from .td
files to Python as appropriate.
> 
I think we should seriously consider doing this.  python would give targets
a lot more flexibility in how they define their instruction sets and I think
it would be much easier to add new features (e.g. patterns for instructions with
condition codes).  Especially, for non-standard targets like AMDGPU which has
had to resort to some pretty creative uses of TableGen to keep the td files
reasonable (see lib/Target/AMDGPU/VOP3Instructions.td).

Another advantage is that I can imagine hardware vendors using python to
generate their own hardware documentation or test suites, so they might be
able to re-use those some tools to generate ISA definitions for LLVM.

-Tom
>  -Hal
> 
>>
>> Example “.td” file:
https://github.com/stoklund/cretonne/blob/master/lib/cretonne/meta/isa/riscv/encodings.py
>>
>> Thanks,
>> /jakob
> 
> -- 
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>