Earlier, I had posted the following question to the group :
> Hello. I have come across a curious result that I cannot explain.
> Hopefully, someone can explain this. I am doing a 1-way ANOVA with 6
> groups (example: summary(aov(y~A)) with A having 6 levels). I get an
> F of 0.899 with 5 and 15 df (p=0.51). I then do the same analysis but
> using data only corresponding to groups 5 and 6. This is, of course,
> equivalent to a t-test. I now get an F of 142.3 with 1 and 3 degrees
> of freedom and a null probability of 0.001. I know that multiple
> comparisons changes the model-wise error rate, but even if I did all
> 15 comparisons of the 6 groups, the Bonferroni correction to a 5%
> alpha is 0.003, yet the Bonferroni correction gives conservative
> rejection levels.
>
> How can such a result occur? Any clues would be helpful.
Brian Ripley, Robert Balshaw, Peter Dalgaard and Ted Harding all
responded. The answer was basically the same from all: If there is
heterogeneity of variances between the groups, and the variances of
groups 5 and 6 are smaller than the others, then my result could occur
because the average within-group variance over all groups in the general
ANOVA is higher than the within-group variance when looking only at
groups 5 and 6. Combine this with the very small sample size and
unequal group membership.
A number of reference books state that ANOVA is fairly robust to
moderate degrees of heterogeneity of variance but not what constitutes
“moderate”!
Bill Shipley
Associate Editor, Ecology
North American Editor, Annals of Botany
Département de biologie, Université de Sherbrooke,
Sherbrooke (Québec) J1K 2R1 CANADA
Bill.Shipley@USherbrooke.ca
<http://callisto.si.usherb.ca:8080/bshipley/>
http://callisto.si.usherb.ca:8080/bshipley/
[[alternative HTML version deleted]]
