The message for aov1 was "Estimated effects <may> be
unbalanced". The
effects are not unbalanced. The design is 'orthogonal'.
The problem is that there are not enough degrees of freedom to estimate
all those error terms. If you change the model to:
aov1 <-
aov(RT~fact1*fact2*fact3+Error(sub/(fact1+fact2+fact3)),data=myData)
or to
aov2 <-
aov(RT~fact1*fact2*fact3+Error(sub/
((fact1+fact2+fact3)^2)),data=myData)
all is well. This last model (aov2) seems to me to have an excessive
number of error terms.
The lme model lme(RT~fact1*fact2*fact3, random=~1|sub, data=myData)
is equivalent to aov0 <- aov(RT~fact1*fact2*fact3+Error(sub),
data=myData)
It can be verified that the estimated variance components can be used
to reproduce the mean squares in the anova table.
A conservative approach is to estimate e.g. contrasts involving fact1
for each subject separately, then comparing these against SE estimates
that have 4df (5 subjects -1). This is the ultimate check -- are
claimed effects consistent against the 5 subjects? The standard errors
should though, probably be based on some variety of averaged variance.
I do not know how to set up the equivalents of aov1 and aov2 (where the
errors are indeed crossed) in lme4.
John Maindonald.
On 12 Aug 2004, at 8:03 PM, r-help-request at stat.math.ethz.ch wrote:
> I will follow the suggestion of John Maindonald and present the
> problem by example with some data.
>
> I also follow the advice to use mean scores, somewhat reluctantly
> though. I know it is common practice in psychology, but wouldnt it be
> more elegant if one could use all the data points in an analysis?
>
> The data for 5 subjects (myData) are provided at the bottom of this
> message. It is a crossed within-subject design with three factors and
> reaction time (RT) as the dependent variable.
>
> An initial repeated-measures model would be:
> aov1<-aov(RT~fact1*fact2*fact3+Error(sub/
> (fact1*fact2*fact3)),data=myData)
>
> Aov complains that the effects involving fact3 are unbalanced:
> > aov1
> ....
> Stratum 4: sub:fact3
> Terms:
> fact3 Residuals
> Sum of Squares 4.81639e-07 5.08419e-08
> Deg. of Freedom 2 8
>
> Residual standard error: 7.971972e-05
>
> 6 out of 8 effects not estimable
> Estimated effects may be unbalanced
> ....
> Presumably this is because fact3 has three levels and the other ones
> only two, making this a 'non-orthogonal design.
>
> I then fit an equivalent mixed-effects model:
> lme1<-lme(RT~fact1*fact2*fact3,data=meanData2,random=~1|sub)
>
> Subsequently I test if my factors had any effect on RT:
> > anova(lme1)
> numDF denDF F-value p-value
>
> (Intercept) 1 44 105294024 <.0001
> fact1 1 44 22 <.0001
> fact2 1 44 7 0.0090
> fact3 2 44 19 <.0001
> fact1:fact2 1 44 9 0.0047
> fact1:fact3 2 44 1 0.4436
> fact2:fact3 2 44 1 0.2458
> fact1:fact2:fact3 2 44 0 0.6660
>
> Firstly, why are the F-values in the output whole numbers?
>
> The effects are estimated over the whole range of the dataset and this
> is so because all three factors are nested under subjects, on the same
> level. This, however, seems to inflate the F-values compared to the
> anova(aov1) output, e.g.
> > anova(aov1)
> ....
> Error: sub:fact2
> Df Sum Sq Mean Sq F value Pr(>F)
> fact2 1 9.2289e-08 9.2289e-08 2.2524 0.2078
> Residuals 4 1.6390e-07 4.0974e-08
> ....
>
> I realize that the (probably faulty) aov model may not be directly
> compared to the lme model, but my concern is if the lme estimation of
> the effects is right, and if so, how can a nave skeptic be convinced
> of this?
>
> The suggestion to use simulate.lme() to find this out seems good, but
> I cant seem to get it working (from "[R] lme: simulate.lme in R"
it
> seems it may never work in R).
>
> I have also followed the suggestion to fit the exact same model with
> lme4. However, format of the anova output does not give me the
> estimation in the way nlme does. More importantly, the degrees of
> freedom in the denominator dont change, theyre still large:
> > library(lme4)
> > lme4_1<-lme(RT~fact1*fact2*fact3,random=~1|sub,data=myData)
> > anova(lme4_1)
> Analysis of Variance Table
>
> Df Sum Sq Mean Sq Denom F value Pr(>F)
> fact1I 1 2.709e-07 2.709e-07 48 21.9205 2.360e-05
> ***
> fact2I 1 9.229e-08 9.229e-08 48 7.4665 0.008772 **
> fact3L 1 4.906e-08 4.906e-08 48 3.9691 0.052047 .
> fact3M 1 4.326e-07 4.326e-07 48 34.9972 3.370e-07 ***
> fact1I:fact2I 1 1.095e-07 1.095e-07 48 8.8619 0.004552 **
> fact1I:fact3L 1 8.988e-10 8.988e-10 48 0.0727 0.788577
> fact1I:fact3M 1 1.957e-08 1.957e-08 48 1.5834 0.214351
> fact2I:fact3L 1 3.741e-09 3.741e-09 48 0.3027 0.584749
> fact2I:fact3M 1 3.207e-08 3.207e-08 48 2.5949 0.113767
> fact1I:fact2I:fact3L 1 2.785e-09 2.785e-09 48 0.2253 0.637162
> fact1I:fact2I:fact3M 1 7.357e-09 7.357e-09 48 0.5952 0.444206
> ---
> I hope that by providing a sample of the data someone can help me out
> on the questions I asked in my previous mail:
>
John Maindonald email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473 fax : +61 2(6125)5549
Centre for Bioinformation Science, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
