llvm dev - Mar 2016 - Head at revision #262824 - breaks Movidius Out-of-Tree target

[I tweaked the subject, #262824 did not introduce the problem, it is just the
version I am first seeing this problem]

A quick update - I have added 'Sched<[]>' as a base class for all
instructions, and also:

    let hasNoSchedulingInfo = 1;

to all the Pseudos, but while most of the errors have gone, I still get the
diagnostic for 'COPY' thus:

    error : No schedule information for instruction 'COPY'

but I don't have an instruction called COPY (this is a target-independent
default).  I also tried adding:

    let CompleteModel = 0;

to my base instruction type, but it is unrecognised.

Thanks,

    MartinO

-----Original Message-----
From: Martin J. O'Riordan [mailto:martin.oriordan at movidius.com] 
Sent: 08 March 2016 15:51
To: 'Tim Northover' <t.p.northover at gmail.com>
Cc: 'LLVM Developers' <llvm-dev at lists.llvm.org>
Subject: RE: [llvm-dev] Head revision #262824 - breaks Movidius Out-of-Tree
target

Thanks Tim,

But I don't understand why it has suddenly become incomplete having been
working fine for years, or what it means by incomplete.  The message simply
says:

    Incomplete schedule models found.
    - Consider setting 'CompleteModel = 0' while developing new models.
    - Pseudo instructions can be marked with 'hasNoSchedulingInfo = 1'.
    - Instructions should usually have Sched<[...]> as a superclass, you
may temporarily use an empty list.

Is there a design document that explains the nature of this change so that I can
better understand the implications?

Our scheduling is very complex, and our implementation still sub-classes
'ScheduleDAGInstrs' (rewriting this is on our TODO list, but it’s a
major undertaking).  We have also had to significantly extend the scheduling
models to handle resources such as multi-ported register access, port-sharing,
instructions that read information from multiple registers at different times,
and which write to different registers at different times.  For example, an
instruction that might read one register at cycle 0, another and cycle 1, then
write a register at cycle 2 and another at cycle 5.  Rewriting this to the
'MISched' model is quite daunting and not something that we can quickly
achieve.

All our instructions use the form:

  let Itinerary = blah { instruction definitions }

The instruction definitions derive from 'Instruction' along the lines
of:

   class SHAVEInstr<args> : Instruction { ... }

but don't have 'Sched<[...]>' as a superclass (should they?).

    MartinO

-----Original Message-----
From: Tim Northover [mailto:t.p.northover at gmail.com] 
Sent: 08 March 2016 15:25
To: Martin J. O'Riordan <Martin.ORiordan at movidius.com>
Cc: LLVM Developers <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Head revision #262824 - breaks Movidius Out-of-Tree
target

Hi Martin,

On 8 March 2016 at 07:15, Martin J. O'Riordan via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> CUSTOMBUILD : error : No schedule information for instruction
'FOOBAR'
I think this is r262384, which turns an incomplete scheduling model into an
error. The commit message gives some suggestions for fixing it.

Cheers.

Tim.

> On Mar 8, 2016, at 8:11 AM, Martin J. O'Riordan via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> 
> [I tweaked the subject, #262824 did not introduce the problem, it is just
the version I am first seeing this problem]
> 
> A quick update - I have added 'Sched<[]>' as a base class for
all instructions, and also:
This should not be necessary if you have itinerary information for your
instructions...
> 
>    let hasNoSchedulingInfo = 1;
> 
> to all the Pseudos, but while most of the errors have gone, I still get the
diagnostic for 'COPY' thus:
> 
>    error : No schedule information for instruction 'COPY'
> 
> but I don't have an instruction called COPY (this is a
target-independent default).  I also tried adding:
COPY instructions exist on any target (they are defined in
include/llvm/Target/Target.td). They definitely occur at the scheduling phase so
it makes sense to provide scheduling information for them. Usually they are
matched with InstRW expressions. The simplest form would be:

def : InstRW<[], (instrs COPY)>

> 
>    let CompleteModel = 0;
> 
> to my base instruction type, but it is unrecognised.
This needs to go into your SchedMachineModel section (it is 1 by default, so you
have to explicitely override it if you want to set it to 0).

- Matthias

Thanks Matthias,

I eventually figured that 'CompleteModel' was supposed to be on the
scheduling model and not the instruction (I grepped the other targets :-) ).

At the moment I have the dummy 'Sched<[]>' base class for my
instructions, I
have added 'CompleteModel = 0' to the schedules (this also cured the
COPY
problem), and 'hasNoSchedulingInfo = 1' to my pseudos and everything
builds
(including libraries), so my next step will be to start the test-suite, wait
all night (I'll test on the simulator) and then see what happened.

So the dummy base 'Sched<[]>' should be unnecessary?  What is this
for, and
if used properly what should be provided in the list?

My schedules specialise 'SchedMachineModel', but I have also had to
extend
TableGen and the information present in the schedules to support some more
complex scheduling information than is currently supported in LLVM.  The
'ItinRW' in particular has extensions to support our architecture.

The reason for this is that I need to be able to say that the register
associated with a particular instruction operand is read or written, using a
particular port (I use 'ProcResource' for these) at a particular cycle
after
the commencement of the instruction.  Most are simple and read at the
beginning of cycle-zero, and write-back at the end of the instruction (it's
natural latency), but there are some more exotic instructions that read at a
later cycle, and write-backs to registers/operands before the instruction
has completed execution (especially pipe-lined instructions), so we extended
the scheduling models to describe this.

Regarding (your previous reply on this topic):

| As of r262384 tablegen checks that it can find scheduling information
| for every instruction. So if you get the message about no information
| about instruction 'FOOBAR', can you given an example how your schedule
| information for FOOBAR looks like? Maybe there is a case missing when
| checking the old itinerary  models.
|
| In the meantime you should be easily able to set:
|
| def MyModel : SchedMachineModel {
|    // ...
|    let CompleteModel = 0;
| }

I'll have to see how to extract this is a readable form - the definitions
use a lot of multi-level 'foreach' and token pasting to construct the
whole
spectrum of port dependencies, so I'll have to manually expand one.  For
example, a write operation is:

    let SchedModel = MyModel in {
        class WriteOp<int latency, ProcResourceKind resource> :
SchedWriteRes<[resource]> {
            let Latency = latency;
                let ResourceDelays = [latency];
        }
    }

A "port" might be described as:

    def 1_Latency_On_Write_Port_4 : WriteOp<1,
!cast<ProcResource>("WP4")>;

Then the itinerary is a list of these, described as:

    def IT_INSN_GROUP_NAME : InstrItinClass;
    def myItin : ItinRW<[1_Latency_On_Write_Port_4, ...],
[IT_INSN_GROUP_NAME]>;

and finally:

    let Itinerary = IT_INSN_GROUP_NAME in { instructions using this
itinerary }

This is a simplification of the actual schedules, but it is structurally
accurate.

But I don't yet understand what it means to have a "complete"
schedule so
that it should not be necessary for me to say:

    let CompleteModel = 0;

I certainly intend that my schedules are accurate and complete (I depend on
them), so if there is something missing then I have the potential for
erroneous scheduling.

Thanks for your help, and to Tim too, I now have it building again and will
know in a few hours if it also works.

	MartinO

-----Original Message-----
From: mbraun at apple.com [mailto:mbraun at apple.com] 
Sent: 08 March 2016 19:13
To: Martin J. O'Riordan <Martin.ORiordan at Movidius.com>
Cc: Tim Northover <t.p.northover at gmail.com>; LLVM Developers
<llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Head at revision #262824 - breaks Movidius
Out-of-Tree target

> On Mar 8, 2016, at 8:11 AM, Martin J. O'Riordan via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> 
> [I tweaked the subject, #262824 did not introduce the problem, it is just
the version I am first seeing this problem]
> 
> A quick update - I have added 'Sched<[]>' as a base class for
all
instructions, and also:
This should not be necessary if you have itinerary information for your
instructions...
> 
>    let hasNoSchedulingInfo = 1;
> 
> to all the Pseudos, but while most of the errors have gone, I still get
the diagnostic for 'COPY' thus:
> 
>    error : No schedule information for instruction 'COPY'
> 
> but I don't have an instruction called COPY (this is a
target-independent
default).  I also tried adding:

COPY instructions exist on any target (they are defined in
include/llvm/Target/Target.td). They definitely occur at the scheduling
phase so it makes sense to provide scheduling information for them. Usually
they are matched with InstRW expressions. The simplest form would be:

def : InstRW<[], (instrs COPY)>

> 
>    let CompleteModel = 0;
> 
> to my base instruction type, but it is unrecognised.
This needs to go into your SchedMachineModel section (it is 1 by default, so
you have to explicitely override it if you want to set it to 0).

- Matthias