llvm dev - May 2018 - [MachineScheduler] Question about IssueWidth / NumMicroOps

Hi Andrew,

Thank you very much for the most helpful explanations! Many things could 
go in as comments, if you ask me - for example:

---
> The LLVM machine model is an abstract machine.
> The abstract pipeline is built around the notion of an "issue
point". This is merely a reference point for counting machine cycles.
>
>
> IssueWidth is meant to be a hard in-order constraint (we sometimes call
this kind of constraint a "hazard"). In the GenericScheduler strategy,
no more than IssueWidth micro-ops can ever be scheduled in a particular cycle.
>
> In practice, IssueWidth is useful to model to the bottleneck between the
decoder (after micro-op expansion) and the out-of-order reservation stations. If
the total number of reservation stations is also a bottleneck, or if any other
pipeline stage has a bandwidth limitation, then that can be naturally modeled by
adding an out-of-order processor resource.
---
> I don't think IssueWidth necessarily has anything to do with
instruction decoding or the execution capacity of functional units. I will say
that we expect the decoding capacity to "keep up with" the issue
width. If the IssueWidth property also serves that purpose for you, I think
that's fine. In the case of the x86 machine models above, since each
instruction is a micro-op, I don't see any useful distinction between decode
bandwidth and in-order issue of micro-ops.
I think this is mostly true for SystemZ also since the majority of 
instructions are basically a single micro-op.
>
> (caveat: there may still be GenericScheduler implementation deficiencies
because it is trying to support more scheduling features than we have in-tree
targets).
Right now it seems that BeginGroup/EndGroup is only used by SystemZ, or? 
I see they are used in checkHazard(), which I actually don't see as 
helpful during pre-RA scheduling for SystemZ. Could this be made 
optional, or perhaps only done post-RA if target does post-RA 
scheduling? SystemZ does post-RA scheduling to manage decoder grouping, 
which is where the BeginGroup/EndGroup and IssueWidth/NumMicroOps is 
useful. However doing this pre-RA and thereby limiting the freedom of 
other heuristics (making less instructions available) seems like a bad idea.
> Sorry, I don't have time to draw diagrams and tables. Hopefully you can
makes sense of my long-form rambling.
Yes, very helpful to me :-)

Thanks again,

Jonas

> On May 14, 2018, at 11:10 AM, Jonas Paulsson <paulsson at
linux.vnet.ibm.com> wrote:
> 
> Hi Andrew,
> 
> Thank you very much for the most helpful explanations! Many things could go
in as comments, if you ask me - for example:
> 
> ---
>> The LLVM machine model is an abstract machine.
> 
>> The abstract pipeline is built around the notion of an "issue
point". This is merely a reference point for counting machine cycles.
>> 
>> 
>> IssueWidth is meant to be a hard in-order constraint (we sometimes call
this kind of constraint a "hazard"). In the GenericScheduler strategy,
no more than IssueWidth micro-ops can ever be scheduled in a particular cycle.
>> 
>> In practice, IssueWidth is useful to model to the bottleneck between
the decoder (after micro-op expansion) and the out-of-order reservation
stations. If the total number of reservation stations is also a bottleneck, or
if any other pipeline stage has a bandwidth limitation, then that can be
naturally modeled by adding an out-of-order processor resource.
> 
> —
https://reviews.llvm.org/D46841 <https://reviews.llvm.org/D46841>
>> I don't think IssueWidth necessarily has anything to do with
instruction decoding or the execution capacity of functional units. I will say
that we expect the decoding capacity to "keep up with" the issue
width. If the IssueWidth property also serves that purpose for you, I think
that's fine. In the case of the x86 machine models above, since each
instruction is a micro-op, I don't see any useful distinction between decode
bandwidth and in-order issue of micro-ops.
> I think this is mostly true for SystemZ also since the majority of
instructions are basically a single micro-op.
> 
>> 
>> (caveat: there may still be GenericScheduler implementation
deficiencies because it is trying to support more scheduling features than we
have in-tree targets).
> Right now it seems that BeginGroup/EndGroup is only used by SystemZ, or? I
see they are used in checkHazard(), which I actually don't see as helpful
during pre-RA scheduling for SystemZ. Could this be made optional, or perhaps
only done post-RA if target does post-RA scheduling? SystemZ does post-RA
scheduling to manage decoder grouping, which is where the BeginGroup/EndGroup
and IssueWidth/NumMicroOps is useful. However doing this pre-RA and thereby
limiting the freedom of other heuristics (making less instructions available)
seems like a bad idea.
I've worked on a few cpus in the past that had issue group restrictions. It
seems like a natural way to handle special kinds of instructions. But I'm
not aware of any LLVM backend that depends on it for preRA scheduling. If they
are, hopefully they're reading this and will speak up.

My thinking a few years back was that targets would only run post-RA scheduling
in rare cases and only for blocks with spill code, as a spill-fixup pass.
That's not what you, and probably others are doing, so if you want to make
those Begin/EndGroup post-RA specific, it's fine with me. Or you could be
more ambitious and introduce the concept of a post-RA specific processor
resource.

-Andy
> 
>> Sorry, I don't have time to draw diagrams and tables. Hopefully you
can makes sense of my long-form rambling.
> Yes, very helpful to me :-)
> 
> Thanks again,
> 
> Jonas
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180514/65ceee9e/attachment.html>

Hi Andy,
>> Right now it seems that BeginGroup/EndGroup is only used by SystemZ, 
>> or? I see they are used in checkHazard(), which I actually don't
see
>> as helpful during pre-RA scheduling for SystemZ. Could this be made 
>> optional, or perhaps only done post-RA if target does post-RA 
>> scheduling? SystemZ does post-RA scheduling to manage decoder 
>> grouping, which is where the BeginGroup/EndGroup and 
>> IssueWidth/NumMicroOps is useful. However doing this pre-RA and 
>> thereby limiting the freedom of other heuristics (making less 
>> instructions available) seems like a bad idea.
>
> I've worked on a few cpus in the past that had issue group 
> restrictions. It seems like a natural way to handle special kinds of 
> instructions. But I'm not aware of any LLVM backend that depends on it 
> for preRA scheduling. If they are, hopefully they're reading this and 
> will speak up.
>
> My thinking a few years back was that targets would only run post-RA 
> scheduling in rare cases and only for blocks with spill code, as a 
> spill-fixup pass. That's not what you, and probably others are doing, 
> so if you want to make those Begin/EndGroup post-RA specific, it's 
> fine with me. Or you could be more ambitious and introduce the concept 
> of a post-RA specific processor resource.
>
This patch: https://reviews.llvm.org/D46870, put those checks under a 
post-RA flag, and also covers the NumUOps / IssueWidth the same way.

/Jonas
> -Andy
>
>>
>>> Sorry, I don't have time to draw diagrams and tables. Hopefully
you
>>> can makes sense of my long-form rambling.
>> Yes, very helpful to me :-)
>>
>> Thanks again,
>>
>> Jonas
>>
>